Behavior recognition model training method and device, electronic equipment, and storage medium

Abstract

A method and device for training a behavior recognition model, an electronic device, and a storage medium are disclosed, relating to the technical field of model training, and including: obtaining a sample image of a driver, and determining a training data set according to the sample image; inputting the training data set into the behavior recognition model for recognition, obtaining a sample recognition result, and determining a recognition confidence according to the sample recognition result and an actual behavior result; if the recognition confidence is less than or equal to a confidence threshold, determining a target loss function value according to the sample recognition result and the actual behavior result, updating the behavior recognition model based on the target loss function value, and training the updated behavior recognition model. The present application can adaptively generate a reorganization kernel according to semantic content, thereby realizing more fine and semantically consistent feature reconstruction, and further ensuring the training accuracy of the behavior recognition model, and further improving the behavior recognition accuracy of the in-vehicle personnel by the trained behavior recognition model.

Description

Technical Field

[0001] This application relates to the field of model training technology, and more specifically, to a training method, apparatus, electronic device, and storage medium for a behavior recognition model. Background Technology

[0002] With the rapid advancement of automotive intelligence and the increasing demands of drivers and passengers for interactive experiences, real-time human-machine interaction and safety monitoring in the in-vehicle environment have become increasingly important. As the core experience unit of a vehicle, the intelligent cockpit's interactive efficiency and safety status directly impact the quality of the driving experience and driving safety. Under the combined influence of complex driving environments and diverse occupant states, the need for accurate understanding of occupant behavior is becoming increasingly prominent. Subtle changes in occupant behavior not only directly affect the smoothness of the interactive experience but may also be linked to critical driving safety conditions. Therefore, improving the accuracy of occupant behavior recognition has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, embodiments of this application propose a training method, apparatus, electronic device, and storage medium for a behavior recognition model to improve the above-mentioned problems.

[0004] According to a first aspect of the embodiments of this application, a training method for a behavior recognition model is provided. The method includes: acquiring sample images of a driver and determining a training dataset based on the sample images, wherein the training dataset includes actual behavior results of the sample images; inputting the training dataset into the behavior recognition model for recognition, and determining kernel weights for different target pixels in each sample image in the training dataset; performing feature recombination in the behavior recognition model based on the kernel weights of different target pixels in each sample image to obtain an upsampled feature map; performing behavior recognition on the upsampled feature map in the behavior recognition model to obtain a sample recognition result, and determining a recognition confidence level based on the sample recognition result and the actual behavior result; if the recognition confidence level is less than or equal to a confidence threshold, determining a target loss function value based on the sample recognition result and the actual behavior result, updating the behavior recognition model based on the target loss function value, and training the updated behavior recognition model.

[0005] In some embodiments, determining the target loss function value based on the sample recognition result includes: determining a first loss value and a second loss value based on the sample recognition result and the actual behavior result; obtaining hyperparameters, wherein the hyperparameters are used to adjust the outlier sensitivity of the sample images in the training dataset; and determining the target loss function value based on the hyperparameters, the first loss value, and the second loss value.

[0006] In some embodiments, determining a first loss value and a second loss value based on the sample recognition result and the actual behavior result includes: determining the recognition anchor box of the sample image based on the sample recognition result and determining the actual anchor box of the sample image based on the actual behavior result; determining the intersection-union ratio (IU) based on the recognition anchor box and the actual anchor box; and determining the first loss value based on the IU, the recognition anchor box, and the actual anchor box.

[0007] In some embodiments, determining the first loss value based on the intersection-union ratio, the identified anchor frame, and the actual anchor frame includes: obtaining a first center point of the identified anchor frame and a second center point of the actual anchor frame; determining a first target distance, a second target distance, and a Euclidean distance between the first center point and the second center point, wherein the first target distance is the distance between the first center point and the second center point on a first coordinate axis, the second target distance is the distance between the first center point and the second center point on a second coordinate axis, and the first coordinate axis and the second coordinate axis are coordinate axes in different directions under the sample image coordinate system; determining an angular cost based on the second target distance and the Euclidean distance; obtaining a first width and a first height of the minimum bounding box in the identified anchor frame and the actual anchor frame, and determining a distance cost based on the first target distance, the second target distance, the angular cost, the first width, and the first height; obtaining a second width and a second height of the identified anchor frame, a third width and a third height of the actual anchor frame, and determining a shape cost based on the second width, the second height, the third width, and the third height; and determining the first loss value based on the distance cost, the shape cost, and the intersection-union ratio.

[0008] In some embodiments, determining the first loss value and the second loss value based on the sample recognition result and the actual behavior result includes: determining the diagonal of the minimum bounding box in the recognition anchor box and the actual anchor box; determining the target parameter based on the diagonal and the Euclidean distance; and determining the second loss value based on the target parameter.

[0009] In some embodiments, the hyperparameters include a first hyperparameter and a second hyperparameter. Determining the target loss function value based on the hyperparameters, the first loss value, and the second loss value includes: determining the mean cross-union ratio (CUP) based on the CUPs of all sample images in the training dataset; determining the outlier coefficient based on the CUP of the current sample image and the mean CUP; determining the target weight based on the outlier coefficient, the first hyperparameter, and the second hyperparameter; and determining the target loss function value based on the target weight, the first loss value, and the second loss value.

[0010] In some embodiments, determining the recognition confidence level based on the sample recognition results and the actual behavior results includes: determining the number of positive detections, the number of false positives, and the number of missed detections based on the sample recognition results and the actual behavior results; determining the accuracy based on the number of positive detections and the number of false positives, and determining the recall based on the number of positive detections and the number of missed detections; determining the average precision of each recognition category identified in the sample recognition results based on the recall; determining the mean average precision based on the number of recognition categories and the average precision corresponding to all recognition categories, and determining the mean average precision as the recognition confidence level.

[0011] According to a second aspect of the embodiments of this application, a training apparatus for a behavior recognition model is provided. The apparatus includes: a sample image acquisition module, configured to acquire sample images of a driver and determine a training dataset based on the sample images, wherein the training dataset includes actual behavior results of the sample images; a kernel weight determination module, configured to input the training dataset into the behavior recognition model for recognition and determine the kernel weight of different target pixels in each sample image in the training dataset; a feature recombination module, configured to reconstruct features in the behavior recognition model based on the kernel weights of different target pixels in each sample image to obtain an upsampled feature map; a recognition confidence module, configured to perform behavior recognition on the upsampled feature map in the behavior recognition model to obtain a sample recognition result and determine a recognition confidence based on the sample recognition result and the actual behavior result; and a training module, configured to determine a target loss function value based on the sample recognition result and the actual behavior result if the recognition confidence is less than or equal to a confidence threshold, update the behavior recognition model based on the target loss function value, and train the updated behavior recognition model.

[0012] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: a processor; and a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the training method for the behavior recognition model as described above is implemented.

[0013] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a processor, implement the training method of the behavior recognition model as described above.

[0014] In this application, the training data determined by the obtained driver sample images is first input into the behavior recognition model for upsampling. This yields the kernel weights of different target pixels in each sample image of the training dataset. Then, the kernel weights of different target pixels in each sample image of the training dataset are used for feature recombination, enhancing the reconstruction capability of subtle behavioral features and dynamic changes, thereby generating a clearer and more expressive upsampled feature map. Furthermore, the behavior recognition model can perform behavior recognition on the upsampled feature map to obtain sample recognition results. Based on the sample recognition results and the actual behavior results of the sample images, the recognition confidence level can be determined. When the recognition confidence level is less than or equal to a confidence threshold, the target loss function value is determined based on the sample recognition results and the actual behavior results. Finally, the target loss function value is used to update the behavior recognition model, and the updated behavior recognition model is then trained. In the embodiments of this application, the recombination kernel can be adaptively generated based on semantic content, thereby achieving more refined and semantically consistent feature reconstruction, ensuring the training accuracy of the behavior recognition model, and improving the accuracy of the trained behavior recognition model in recognizing the behavior of people inside the vehicle.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit the embodiments of this application. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] Figure 1 This is a schematic diagram of a behavior recognition system according to an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating a training method for a behavior recognition model according to an embodiment of this application.

[0019] Figure 3 This is a flowchart illustrating a training method for a behavior recognition model according to another embodiment of this application.

[0020] Figure 4 This is a flowchart illustrating the specific steps of step 350 according to an embodiment of this application.

[0021] Figure 5 This is a flowchart illustrating the specific steps of step 430 according to an embodiment of this application.

[0022] Figure 6 This is a flowchart illustrating the specific steps of step 370 according to an embodiment of this application.

[0023] Figure 7 This is a flowchart illustrating a training method for a behavior recognition model according to another embodiment of this application.

[0024] Figure 8 This is a block diagram of a training apparatus for a behavior recognition model according to an embodiment of this application.

[0025] Figure 9 This is a hardware structure diagram of an electronic device according to an embodiment of this application.

[0026] The accompanying drawings have illustrated specific embodiments of the present application. More detailed descriptions will follow. These drawings and descriptions are not intended to limit the scope of the present application's embodiments in any way, but rather to illustrate the concepts of the present application's embodiments to those skilled in the art through specific embodiments. Detailed Implementation

[0027] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0028] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0029] Please see Figure 1 , Figure 1 An embodiment of the behavior recognition system provided in this application is illustrated, such as... Figure 1 As shown below, the training method for implementing the behavior recognition model in the behavior recognition system will be illustrated by example.

[0030] In one alternative implementation, the behavior recognition system includes an image acquisition module 110, a processing module 120, and a display module 130, wherein the processing module 120 refers to a software unit or module, and the image acquisition module 110 and the display module 130 refer to hardware devices.

[0031] For example, the image acquisition module 110 acquires sample images of the driver and then sends the sample images to the processing module 120. The processing module 120 acquires the sample images of the driver and determines the training dataset based on the sample images. The training dataset is input into the behavior recognition model for recognition to obtain sample recognition results. The sample recognition results and the corresponding training dataset are then sent to the display module 130. The display module 130 displays the sample recognition results, sample images, and actual behavior results of the sample images. The processing module 120 determines the recognition confidence based on the sample recognition results and actual behavior results displayed in the display module 130. Based on this, if the recognition confidence is less than or equal to a confidence threshold, the target loss function value is determined based on the sample recognition results and actual behavior results. The behavior recognition model is then updated based on the target loss function value, and the updated behavior recognition model is trained.

[0032] Figure 1 The system in [the document] can be used to implement the following Figure 2 For the training method of the described behavior recognition model, please refer to [link / reference]. Figure 2 , Figure 2 This application illustrates a method for training a behavior recognition model according to an embodiment of the present application. In a specific embodiment, this method for training the behavior recognition model can be applied to, for example... Figure 8 The training device 700 for the behavior recognition model shown, and the electronic device 800 equipped with the training device 700 for the behavior recognition model. Figure 9 The specific process of this embodiment will be described below. Of course, it is understood that this method can be executed by an electronic device with computing power, such as a vehicle-mounted server, a cloud server, or other processors. The following will focus on... Figure 2 The process shown is described in detail. The training method of the behavior recognition model may specifically include the following steps 210-250.

[0033] Step 210: Obtain sample images of the driver and determine a training dataset based on the sample images, wherein the training dataset includes the actual behavioral results of the sample images.

[0034] As an alternative approach, subtle changes in the behavior of occupants, especially the driver, during vehicle operation not only directly affect the smoothness of the interactive experience but may also be related to critical driving safety conditions (such as fatigue driving warnings and dangerous emotion recognition). Therefore, in order to ensure safe vehicle operation, it is necessary to identify the driver's behavior, determine the vehicle's driving safety level based on the identified driver behavior, and then determine the corresponding driving strategy based on the vehicle's driving safety level.

[0035] In one alternative scenario, to ensure accurate recognition of driver behavior, the behavior recognition model can be trained beforehand. This allows the trained model to be directly used in offline applications to identify the behavior of occupants inside the vehicle. Therefore, sample images of the driver inside the vehicle can be acquired first, and these sample images can be used to determine the training dataset for training the behavior recognition model.

[0036] Optionally, after obtaining the sample images, the actual behavior results of the drivers in the sample images can be labeled, or each sample image and its corresponding actual behavior result can be identified by a unique identifier to obtain a training dataset.

[0037] Optionally, an image acquisition device, such as an in-vehicle camera, can be installed in the vehicle to monitor the driver's face around the clock, focusing on capturing typical behaviors such as normal driving, fatigue, distraction, and emotional fluctuations, thereby obtaining sample images of the driver. The image acquisition device must have high dynamic range and low-light enhancement performance to ensure clear acquisition of facial details even under conditions of drastic lighting changes, such as backlighting, nighttime, or entering / exiting tunnels.

[0038] Optionally, the sample images in the training dataset should cover the complete spectrum of subtle behaviors and include behavioral characteristics of different age groups, genders, skin colors, etc., under various lighting conditions, occlusions, and head poses. The acquired sample images should also comprehensively consider the influence of multiple factors such as day / night cycles, seasonal climate, and changes in the in-vehicle environment to enhance the adaptability of the behavior recognition model in real-world driving scenarios.

[0039] In one alternative scenario, to ensure the generalization performance and system stability of the behavior recognition model, sample images can be preprocessed using data augmentation methods such as image rotation, scaling, mirroring, brightness adjustment, and noise addition to construct diverse sample images and obtain a training dataset.

[0040] Optionally, during the preprocessing of sample images, the data quality of the sample images can be ensured by removing blurry, severely occluded, or invalid image frames. Furthermore, the LabelImg tool can be used to annotate the sample images, focusing on the driver's facial area and their behavioral categories.

[0041] Step 220: Input the training dataset into the behavior recognition model for recognition, and determine the kernel weights of different target pixels in each sample image in the training dataset.

[0042] As an alternative approach, after determining the training dataset, the training dataset can be used as input data into the behavior recognition model to be trained. The behavior recognition model can then recognize each sample image in the training dataset.

[0043] In one alternative scenario, when an action recognition model identifies sample images, the backbone network of the model continuously downsamples through convolution and pooling, resulting in increasingly smaller feature maps. However, this makes it difficult to accurately identify the specific location of objects within the sample images. To accurately identify the location of objects in sample images, upsampling can be used to enlarge deep, semantically rich, but low-resolution feature maps. These upsampling maps can then be fused with high-resolution but semantically simple feature maps, resulting in feature maps with rich semantic information and precise location information.

[0044] In related technologies, upsampling is typically performed using methods such as nearest neighbor / bilinear interpolation and deconvolution. However, these upsampling methods do not utilize the semantic content of the feature maps in the sample images, leading to inaccurate behavior recognition in the images. Therefore, a unique upsampling kernel can be dynamically and adaptively generated for each upsampling location in the sample image based on the semantic content of its surrounding neighborhood. Then, features in the sample image can be reconstructed based on this kernel, thereby more accurately and efficiently recovering spatial details beneficial to downstream tasks (such as detection and segmentation).

[0045] Optionally, features can be extracted from the input sample image to obtain a feature map, which is then convolved to compress the number of channels. Next, for each target location in the output feature map, the features of its corresponding neighborhood in the input feature map are determined. Finally, the extracted neighborhood features are input into a small convolutional network (usually two convolutional layers) in the behavior detection model to generate a corresponding prediction kernel, thus obtaining the kernel weights for different target pixels in each sample image of the training dataset. The prediction kernel is dynamically generated; it differs for different locations in the output feature map, and its generation depends entirely on the content of the target pixel's location in the image's corresponding neighborhood in the input feature map.

[0046] Step 230: Based on the kernel weights of different target pixels in each sample image, feature recombination is performed in the behavior recognition model to obtain an upsampled feature map.

[0047] As an alternative approach, after determining the kernel weights of different target pixels in each sample image of the training dataset, the source regions corresponding to the positions of different target pixels (which can be regions centered around a certain input position) can be determined in the feature map of the sample image. Then, the kernel weights of different target pixels in each sample image and the source regions corresponding to the positions of different target pixels are weighted and summed to obtain the feature values ​​of the positions of different target pixels, thereby achieving feature recombination and obtaining the upsampled feature map.

[0048] Step 240: Perform behavior recognition on the upsampled feature map in the behavior recognition model to obtain sample recognition results, and determine the recognition confidence level based on the sample recognition results and the actual behavior results.

[0049] As an alternative approach, after obtaining the upsampled feature map of the sample image, the detection head in the behavior recognition model can be used to identify different objects in the upsampled feature map, thereby obtaining the sample recognition result corresponding to each sample image. The sample recognition result for each sample image includes the anchor box for each person in the sample image and the identified behavior of each person.

[0050] In one alternative scenario, after obtaining the sample recognition results, statistics can be performed on the sample recognition results and the actual behavior results of the sample images to determine the number of positive detections, the number of missed detections, and the number of false detections of the sample recognition model in the training dataset. Then, the recognition confidence of the behavior recognition model can be determined based on the number of positive detections, the number of missed detections, and the number of false detections, and the recognition accuracy in the behavior recognition model can be determined based on the recognition confidence.

[0051] Step 250: If the recognition confidence is less than or equal to the confidence threshold, then determine the target loss function value based on the sample recognition result and the actual behavior result, update the behavior recognition model based on the target loss function value, and train the updated behavior recognition model.

[0052] As an alternative approach, after determining the recognition confidence level of the behavior recognition model, the recognition confidence level can be compared with the confidence level threshold to determine whether the recognition accuracy of the behavior recognition model meets the corresponding requirements, and then to determine whether to continue training the behavior recognition model.

[0053] In one optional scenario, when the recognition confidence is determined to be less than or equal to the confidence threshold, it can be determined that the recognition accuracy of the behavior recognition model does not meet the corresponding requirements. At this time, the accuracy of the behavior recognition model in recognizing the behavior of people in the image is relatively low. Therefore, it can be determined that the parameters in the behavior recognition model need to be adjusted, and the behavior recognition model is trained again based on the adjusted model until the recognition confidence is determined to be greater than the confidence threshold, and the training of the behavior recognition model ends.

[0054] Optionally, when it is determined that the behavior recognition model still needs to be trained, in order to improve the training efficiency and accuracy of the behavior recognition model, the target loss function value can be determined first based on the sample recognition results and the actual behavior results, and then the parameters in the behavior recognition model can be adjusted based on the target loss function value, thereby updating the behavior recognition model.

[0055] Optionally, during the process of the behavior recognition model recognizing the sample image, the position of the object in the corresponding sample image is one of the important factors of recognition accuracy. The position difference of the object in the sample image can be used as the target loss function. In this way, the target loss function value is determined by determining the position difference between the position of the object in the sample recognition result and the position of the object in the sample image in the actual behavior result.

[0056] In one alternative scenario, in order to accurately determine the target loss function value, the intersection-union ratio of the first anchor box and the second anchor box can be determined by determining the first anchor box corresponding to the position of the identified object in the sample recognition result and the second anchor box corresponding to the position of the identified object in the sample image in the actual behavior result, thereby determining the position difference.

[0057] Optionally, since the Cross-Union Ratio (CIRR) measures the overlap between the first and second anchor boxes and does not include the influence of geometric factors, the target loss function value can be comprehensively determined by considering the aspect ratio consistency and center point distance between the first and second anchor boxes based on the CIRR, thus ensuring the accuracy of the model's training on the behavior.

[0058] Optionally, however, its performance still faces challenges in vehicle driver behavior detection scenarios. For example, the target loss function value determined by the first and second anchor boxes does not sufficiently penalize the center point offset and scale error of small targets, resulting in limited localization accuracy. Furthermore, complex background thermal noise and significant scale differences between target objects can easily cause the model to develop a learning bias towards the dominant category, thereby increasing the risk of missed detections. To address these issues, a composite loss value can be determined by combining geometric constraint enhancement strategies and dynamic focusing mechanisms. This composite loss value is then used as the target loss value to update the behavior recognition model. This composite loss not only adaptively allocates gradient gains to balance the learning of targets at different scales but also reduces the optimization degrees of freedom by introducing additional geometric constraints. This significantly improves the localization robustness and classification reliability of small targets while maintaining the detection accuracy of large targets.

[0059] In the embodiments of this application, training data determined by the acquired driver sample images is first input into the behavior recognition model for upsampling, thereby obtaining the kernel weights of different target pixels in each sample image of the training dataset. Then, feature recombination is performed using the kernel weights of different target pixels in each sample image of the training dataset to enhance the reconstruction capability of subtle behavioral features and dynamic changes, thereby generating a clearer and more expressive upsampled feature map. Furthermore, the behavior recognition model can perform behavior recognition on the upsampled feature map to obtain sample recognition results. Based on the sample recognition results and the actual behavior results of the sample images, the recognition confidence can be determined. When the recognition confidence is less than or equal to the confidence threshold, the target loss function value can be determined based on the sample recognition results and the actual behavior results. Finally, the target loss function value is used to update the behavior recognition model, and the updated behavior recognition model is trained. In the embodiments of this application, the recombination kernel can be adaptively generated according to the semantic content, thereby achieving more refined and semantically consistent feature reconstruction, thus ensuring the training accuracy of the behavior recognition model, and improving the accuracy of the behavior recognition model for identifying people in the vehicle.

[0060] Please see Figure 3 , Figure 3 This paper illustrates a training method for a behavior recognition model provided in an embodiment of this application. The following will focus on... Figure 3 The process shown is described in detail. The training method of the behavior recognition model may specifically include the following steps 310-370.

[0061] Step 310: Obtain sample images of the driver and determine a training dataset based on the sample images, wherein the training dataset includes the actual behavioral results of the sample images.

[0062] Step 320: Input the training dataset into the behavior recognition model for recognition, and determine the kernel weights of different target pixels in each sample image in the training dataset.

[0063] Step 330: Based on the kernel weights of different target pixels in each sample image, feature recombination is performed in the behavior recognition model to obtain an upsampled feature map.

[0064] Step 340: Perform behavior recognition on the upsampled feature map in the behavior recognition model to obtain sample recognition results, and determine the recognition confidence level based on the sample recognition results and the actual behavior results.

[0065] The specific steps of steps 310-340 can be found in steps 210-240, and will not be repeated here.

[0066] Step 350: If the recognition confidence is less than or equal to the confidence threshold, then determine the first loss value and the second loss value based on the sample recognition result and the actual behavior result.

[0067] As an alternative approach, when the recognition confidence is less than or equal to the confidence threshold, it can be determined that the recognition accuracy of the behavior recognition model cannot meet the requirements of offline applications. Therefore, the behavior recognition model needs to be updated. The first loss value and the second loss value can be determined based on the sample recognition results and the actual behavior results, respectively. In this way, the target loss function value can be determined based on the first loss value and the second loss value. Finally, the behavior recognition model can be updated based on the target loss function value.

[0068] Optionally, the first loss value can be the loss value determined according to the standardized intersection-union loss function. In the first loss value, the angular cost constraint is added to the anchor box corresponding to the recognition object in the sample image, thereby guiding the behavior recognition model to optimize along a more efficient path.

[0069] Optionally, the second loss value can be a loss value determined by a dynamically non-monotonic focused cross-union ratio loss function. In the second loss value, the contribution weight of different recognition objects to the loss is dynamically adjusted according to the regression quality of the bounding box, so that the model focuses more on improving samples of ordinary quality, rather than being dominated by recognition objects that are extremely difficult or extremely easy to recognize.

[0070] In some embodiments, such as Figure 4 As shown, step 350 includes steps 410-430.

[0071] Step 410: Determine the recognition anchor frame of the sample image based on the sample recognition result and determine the actual anchor frame of the sample image based on the actual behavior result.

[0072] As an alternative approach, to improve the accuracy of driver behavior recognition and the recognition of images or objects at different scales, when the behavior recognition model identifies sample images, it can first determine the bounding box of the object in the image. This allows the behavior recognition model to directly identify the semantic information within the bounding box, thereby obtaining the behavior category of the person in the sample image. Therefore, the bounding box of the object in the sample image significantly affects the recognition result of the behavior recognition model. Furthermore, to improve the recognition accuracy of the behavior recognition model, the bounding boxes in the sample recognition results and the bounding boxes in the actual behavior results can be used as benchmarks to determine the target loss function value.

[0073] Optionally, in order to accurately distinguish between the bounding boxes of the sample images in the sample recognition results and the actual anchor boxes of the sample images in the actual behavior results, the bounding boxes in different results can be labeled with bounding boxes of different pixel values, thereby obtaining the recognition anchor boxes and the actual anchor boxes respectively.

[0074] Step 420: Determine the intersection-union ratio based on the identified anchor frame and the actual anchor frame.

[0075] As an optional approach, after determining the identification anchor frame and the actual anchor frame, the first coordinate of the top-left corner and the second coordinate of the top-right corner of the identification anchor frame in the sample image can be determined, as well as the third coordinate of the top-left corner and the fourth coordinate of the top-right corner of the actual anchor frame in the sample image. Then, based on the first, second, third, and fourth coordinates, the intersection and union regions between the identification anchor frame and the actual anchor frame can be determined. This allows the intersection-union ratio (CUIR) between the identification anchor frame and the actual anchor frame to be determined. The CUIR ranges from [0,1]. A CUIR of 0 indicates that there is no overlap between the identification anchor frame and the actual anchor frame, while a CUIR of 1 indicates that the identification anchor frame and the actual anchor frame completely overlap.

[0076] In one optional scenario, the top-left corner coordinate of the intersection can be determined by intersecting the first and third coordinates, and the top-right corner coordinate can be determined by intersecting the second and fourth coordinates. The identified anchor frame and the actual anchor frame only intersect when the x-coordinate of the bottom-right corner is greater than the x-coordinate of the top-left corner, and the y-coordinate is greater than the y-coordinate of the top-left corner. Optionally, if there is no intersection between the identified anchor frame and the actual anchor frame, the area of ​​their intersection region is 0; if there is an intersection between the identified anchor frame and the actual anchor frame, the intersection region can be determined by calculating the area of ​​the overlapping portion between them.

[0077] Optionally, the first area of ​​the identified anchor frame and the second area of ​​the actual anchor frame can be determined separately. Then, the area of ​​the union region can be determined based on the first area, the second area, and the area of ​​the intersection region. Finally, the intersection-union ratio between the identified anchor frame and the actual anchor frame can be determined by determining the area of ​​the intersection region and the area of ​​the union region. Optionally, the area of ​​the union region can be determined by first determining the sum of the first and second areas, and then determining the area difference between the sum of the areas and the area of ​​the intersection region.

[0078] Step 430: Determine the first loss value based on the crossover ratio, the identified anchor frame, and the actual anchor frame.

[0079] As an alternative approach, after determining the Cross-Union Ratio (CUI), in order to accurately determine the target loss function value, a first loss value can be determined by the CUI, the coordinate information of the identified anchor boxes in the sample image, and the coordinate information of the actual anchor boxes in the sample image. The target loss function value can then be determined based on the first loss value.

[0080] In one alternative scenario, since the intersection-union ratio (IU) only reflects the degree of overlap between the identified anchor frame and the actual anchor frame, and cannot reflect the differences in distance, angle, and shape between the identified anchor frame and the actual anchor frame, the determined first loss value is not accurate enough, which in turn leads to an inaccurate determined target loss function value. Therefore, the first loss value can be determined by combining the distance cost, angle cost, and shape cost between the identified anchor frame and the actual anchor frame, thereby incorporating the IU, distance cost, angle cost, and shape cost.

[0081] In some embodiments, such as Figure 5 As shown, step 430 further includes steps 431-436.

[0082] Step 431: Obtain the first center point of the identified anchor frame and the second center point of the actual anchor frame.

[0083] As an alternative approach, in order to determine the distance cost between the identified anchor frame and the actual anchor frame, the first center point of the identified anchor frame and the second center point of the actual anchor frame can be obtained first. This allows the corresponding distance cost to be determined based on the position information of the first center point and the second center point in the sample image.

[0084] Step 432: Determine the first target distance, the second target distance, and the Euclidean distance between the first center point and the second center point, wherein the first target distance is the distance between the first center point and the second center point on the first coordinate axis, the second target distance is the distance between the first center point and the second center point on the second coordinate axis, and the first coordinate axis and the second coordinate axis are coordinate axes in different directions under the sample image coordinate system.

[0085] As an alternative approach, after determining the first center point and the second center point, the first coordinate information of the first center point in the image coordinate system of the sample image and the second coordinate information of the second center point in the same coordinate system can be determined first. Then, the first target distance and the second target distance can be determined based on the first coordinate information and the second coordinate information.

[0086] In one alternative scenario, the first target distance can be determined by determining a first difference between the coordinate values ​​of the first coordinate axis in the first coordinate information and the coordinate values ​​of the first coordinate in the second coordinate information, and the second target distance can be determined by determining a second difference between the coordinate values ​​of the second coordinate axis in the first coordinate information and the coordinate values ​​of the second coordinate in the second coordinate information. Optionally, the Euclidean distance between the first center point and the second center point can be determined using the Euclidean distance calculation formula.

[0087] Step 433: Determine the angle cost based on the second target distance and the Euclidean distance.

[0088] As an optional approach, the angle cost is calculated as the vector angle between the center points of the identified anchor box and the actual anchor box. This angle guides the model to prioritize aligning the bounding box, thereby aligning the predicted box to the horizontal or vertical direction of the true bounding box as quickly as possible. Therefore, the angle cost can be determined first based on the second target distance and the Euclidean distance. Alternatively, it can be determined using the formula... To determine the angle cost, where, , For angle cost, The distance to the second target. It is a Euclidean distance.

[0089] Step 434: Obtain the first width and first height of the minimum bounding box between the identified anchor frame and the actual anchor frame, and determine the distance cost based on the first target distance, the second target distance, the angle cost, the first width, and the first height.

[0090] As an alternative approach, since the minimum bounding box provides a scale-invariant penalty in the distance cost, ensures a smooth transition of distance metrics during the separation and overlap of the identification anchor box and the actual anchor box, and the diagonal of the minimum bounding box still provides a valid distance metric when the identification anchor box and the actual anchor box are in an containment relationship, it is necessary to determine the minimum bounding box and its first width and first height within the identification anchor box and the actual anchor box.

[0091] In one alternative scenario, a first index can be determined based on the first target distance and the first width, and a second index can be determined based on the second target distance and the first height. Then, a first cost can be determined based on the first index and the angle cost, and a second cost can be determined based on the second index and the angle cost. Finally, the first cost and the second cost are summed to obtain the distance cost. Alternatively, this can be achieved using a formula... To determine the distance cost, where, For distance cost, It can be either the first index or the second index, the first index Through formula Confirmed, second index Through formula Sure, The distance to the first target. The first width, It is the highest altitude.

[0092] In another alternative scenario, the square of the Euclidean distance between the first and second center points can be calculated first. Then, the diagonal length of the minimum bounding box in the identified anchor frame and the actual anchor frame can be calculated. Finally, the distance cost can be determined by the diagonal length, the square of the Euclidean distance, and the angle cost. The distance cost can be adjusted using the angle cost so that the distance cost is reduced when the angle is well aligned.

[0093] Step 435: Obtain the second width and second height of the identified anchor frame, the third width and third height of the actual anchor frame, and determine the shape cost based on the second width, second height, third width and third height.

[0094] As an alternative approach, shape cost measures the similarity in shape between two bounding boxes, typically calculated based on differences in width and height. Therefore, it's necessary to obtain the second width and second height of the identified anchor box, as well as the third width and third height of the actual anchor box, to determine the shape cost based on these dimensions. The shape cost primarily penalizes differences in shape (aspect ratio) between the predicted and ground truth bounding boxes.

[0095] In one alternative scenario, the relative difference between the width and height of the identified anchor frame and the actual anchor frame can be determined first, and then the shape cost can be determined based on the relative difference between the width and height of the identified anchor frame and the actual anchor frame.

[0096] Optionally, the relative height difference between the identified anchor frame and the actual anchor frame can be determined first based on the second and third heights, and the relative width difference between the identified anchor frame and the actual anchor frame can be determined based on the second and third widths. Then, the shape cost is determined based on the relative height and width differences. This can be achieved using the formula... Determine the relative height difference using the formula Determine the relative differences in width, and finally base it on the formula. Determine the shape cost, where, For shape cost, w is the second width. h is the third width, and h is the second height. This is the third width.

[0097] Step 436: Determine the first loss value based on the distance cost, the shape cost, and the intersection-union ratio.

[0098] As an optional approach, to improve the training efficiency and recognition accuracy of the behavior recognition model, the overlap penalty, position penalty, and shape penalty between the identified anchor box and the actual anchor box can be comprehensively used to train and update the behavior recognition model. This ensures that when the behavior recognition model recognizes an image, it maintains a basic overlap between the identified anchor box and the actual bounding box of the object, achieving intelligent position alignment and ensuring shape similarity, thereby improving the recognition accuracy of the behavior recognition model. Optionally, this can be achieved through formula... To determine the first loss value, where, The first loss value, The intersection-union ratio loss can be calculated using the formula... Confirmed, IoU is the intersection-union ratio.

[0099] In some embodiments, step 350 further includes: determining the diagonal of the minimum bounding box in the identification anchor frame and the actual anchor frame; determining the target parameter based on the diagonal and the Euclidean distance; and determining the second loss value based on the target parameter.

[0100] As an alternative approach, to avoid excessive competition for anchor boxes during the training process of the behavior recognition model, as well as the harmful gradients introduced by low-quality samples, which lead to insufficient training stability and generalization performance of the behavior recognition model, a gradient allocation strategy can be introduced. The quality of the anchor boxes can be measured by outlier, thereby determining the second loss value. The target loss function value can then be determined in combination with the second loss value, so that the behavior recognition model can be trained and updated using the target loss function value.

[0101] In one alternative scenario, since low-quality sample images generate large gradients during training, these gradients may be noisy or misleading, leading to unstable model training, overfitting, and impaired generalization ability. Therefore, the target loss function value can be determined by comprehensively considering the loss values ​​of outliers. To accurately determine the target loss function value, a dynamic weighting factor for adjusting outlier count can be first determined. This dynamic weighting factor can then be used as a second loss value, and the target loss function value can be determined based on this second loss value.

[0102] Alternatively, the second loss value can be determined by calculating a dynamic weighting factor related to the size of the anchor frame, or by determining the variance of the distance difference between the identified anchor frame and the actual anchor frame. This can be achieved using a formula... To determine, among which, This is the second loss value.

[0103] Please continue reading. Figure 3 Step 360: Obtain hyperparameters, wherein the hyperparameters are used to adjust the outlier sensitivity of the sample images in the training dataset.

[0104] In order to adaptively adjust the gradient gain allocation strategy when training the behavior recognition model and thus maintain a more reasonable gradient control throughout the training process, one can first obtain the hyperparameters used to adjust the outlier sensitivity, and then determine the target loss function value based on the hyperparameters.

[0105] Step 370: Determine the target loss function value based on the hyperparameters, the first loss value, and the second loss value.

[0106] As an alternative approach, the target loss function value can be determined by measuring the product of the hyperparameters, the first loss value, and the second loss value. For example, it can be determined using the formula... To determine the target loss function value, where, The target loss function value is r, which is a hyperparameter. The second loss value will significantly amplify the first loss value of the ordinary quality recognition anchor boxes of the sample images in the training dataset.

[0107] In some embodiments, such as Figure 6 As shown, the hyperparameters include a first hyperparameter and a second hyperparameter, and step 370 includes steps 510-540.

[0108] Step 510: Determine the mean cross-union ratio (CUP) based on the CUP corresponding to all sample images in the training dataset.

[0109] As an alternative approach, to accurately determine the target loss function value, the average loss level representing the sample images in the current training phase can be determined first. This average loss level can then be used to determine the outlier degree of each sample image during the training phase, and subsequently, the target loss function value can be determined based on the outlier degree. Therefore, the mean Intersection over Union (IoU) ratio can be determined first, and this mean IoU ratio can be used as the average loss level to determine the outlier degree against the IoU ratio corresponding to each sample image. Alternatively, the mean IoU ratio for the current training iteration can be determined based on the number of sample images in each input training dataset, the IoU ratio of each sample image, the mean IoU ratio updated after each training iteration, and the mean IoU ratio from the previous training iteration.

[0110] Step 520: Determine the cross-union ratio (CUP) corresponding to the current sample image and the mean CUP to determine the outlier coefficient.

[0111] As an alternative approach, outlier measure can assess the degree of deviation between the current sample and the normal samples. Therefore, the outlier coefficient can be determined by comparing the cross-union ratio (CUI) of the current sample image with its mean CUI. This outlier coefficient can then be used to determine the outlier degree of the training sample images. Alternatively, it can be determined using the formula... To determine the outlier coefficient, where, This is the outlier coefficient. The intersection-union ratio (IU) of the current sample image. The intersection-union ratio is the mean. The asterisk (*) indicates a separation operation, meaning that during backpropagation, the labeled variable is only used as a reference for calculating outlier degree and does not participate in gradient updates.

[0112] Step 530: Determine the target weight based on the outlier coefficient, the first hyperparameter, and the second hyperparameter.

[0113] As an alternative approach, after determining the outlier coefficient, the target weight can be determined based on the outlier coefficient, the first hyperparameter, and the second hyperparameter. Alternatively, it can be determined using a formula. To determine the target weight, where r is the target weight. As the first hyperparameter, This is the second hyperparameter.

[0114] Step 540: Determine the target loss function value based on the target weight, the first loss value, and the second loss value.

[0115] As an alternative approach, the target loss function value can be determined directly by multiplying the target weight, the first loss value, and the second loss value. This composite loss not only adaptively allocates gradient gains to balance the learning of targets at different scales, but also reduces the optimization degrees of freedom by introducing additional geometric constraints. This significantly improves the robustness of localization and the reliability of classification for weak targets while maintaining high detection accuracy.

[0116] In this embodiment, a first loss value and a second loss value are first determined based on the sample recognition results and the actual behavior results. Then, a target loss function value is determined based on the hyperparameter used to adjust the outlier sensitivity of sample images in the training dataset, the first loss value, and the second loss value. The behavior recognition model is then trained and updated using this target loss function value. This embodiment significantly improves the accuracy and robustness of the behavior recognition model in recognizing driver behavior by optimizing the matching strategy between the recognized anchor boxes and the actual anchor boxes and by strengthening the focus on general-quality samples.

[0117] Please see Figure 7 , Figure 7 This paper illustrates a training method for a behavior recognition model provided in an embodiment of this application. The following will focus on... Figure 7 The process shown is described in detail. The training method of the behavior recognition model may specifically include the following steps 610-680.

[0118] Step 610: Obtain sample images of the driver and determine a training dataset based on the sample images, wherein the training dataset includes the actual behavioral results of the sample images.

[0119] Step 620: Input the training dataset into the behavior recognition model for recognition, and determine the kernel weights of different target pixels in each sample image in the training dataset.

[0120] Step 630: Based on the kernel weights of different target pixels in each sample image, feature recombination is performed in the behavior recognition model to obtain an upsampled feature map.

[0121] Step 640: Perform behavior recognition on the upsampled feature map in the behavior recognition model to obtain sample recognition results, and determine the number of positive detections, false detections, and missed detections based on the sample recognition results and the actual behavior results.

[0122] As an alternative approach, to avoid overtraining the behavior recognition model and resulting in low recognition accuracy, the number of correct positive detections, the number of false positives, and the number of missed detections can be determined based on the sample recognition results and the actual behavior results. Specifically, the number of correct positive detections refers to the number of true positives (TP) in each sample image where the identified behavior is identical to the behavior in the actual behavior result; the number of false positives (FP) refers to the number of false positives (FP) in each sample image where the identified behavior is different from the behavior in the actual behavior result; and the number of false negatives (FN) refers to the number of false negatives (FN) in each sample image where the behavior exists in the actual behavior result but was not recognized in the sample recognition results.

[0123] Step 650: Determine the accuracy rate based on the number of positive detections and the number of false positives, and determine the recall rate based on the number of positive detections and the number of missed detections.

[0124] As an alternative approach, after determining the number of positive detections, false positives, and missed detections, one can first determine the first sum of the number of positive and missed detections, then determine the first ratio between the number of positive detections and the first sum, and finally determine the first ratio as the accuracy rate. This can be achieved using a formula. To determine the accuracy, P is the accuracy, TP is the number of positive detections, and FP is the number of false positives.

[0125] Optionally, a second sum of the number of positive detections and the number of negative detections can be determined first, and then a second ratio of the number of positive detections to the second sum can be determined. This second ratio can then be used to determine the recall rate. This can be achieved using a formula. Precision is determined by R, where R is the recall and FN is the number of false negatives. Recall is the proportion of positive detections out of all true positives in the training dataset, thus measuring the fullness of detection.

[0126] Step 660: Determine the average precision of each identified category in the sample identification results based on the recall rate.

[0127] As an alternative approach, after determining the recall rate, to determine the recognition accuracy of the behavior recognition model on the training dataset, the average precision of each recognized category in the sample recognition results can be determined by the area under the curve plotted with recall on the horizontal axis and precision on the vertical axis. Alternatively, this can be achieved using the formula... The average accuracy of each identified category in the sample recognition results is determined. A higher average accuracy indicates better performance of the corresponding behavior recognition model.

[0128] Step 670: Determine the mean average precision based on the number of categories of the identification category and the average precision corresponding to all identification categories, and determine the mean average precision as the identification confidence level.

[0129] As an optional approach, to ensure the training accuracy of the behavior recognition model, it is necessary to ensure that the behavior recognition model performs well in recognizing different categories of behaviors. This can be achieved by taking the average accuracy of the behavior recognition model across all behavior categories when recognizing sample images, and using this average to determine the recognition confidence of the behavior recognition model. Alternatively, it can be determined using a formula... To determine the mean average precision, where mAP is the mean average precision and n is the number of behavior categories in all sample images in the training dataset.

[0130] Step 680: If the recognition confidence is less than or equal to the confidence threshold, then determine the target loss function value based on the sample recognition result and the actual behavior result, update the behavior recognition model based on the target loss function value, and train the updated behavior recognition model.

[0131] The specific steps of steps 610-630 and 680 can be found in steps 210-230 and 250, and will not be repeated here.

[0132] In this embodiment, the number of positive detections, false positives, and missed detections are first determined based on the sample recognition results and the actual behavior results. This allows for the determination of accuracy based on the number of positive and false positives, and recall based on the number of positive and missed detections. Subsequently, the average precision of each recognition category identified in the sample recognition results can be determined based on the recall. Finally, the mean average precision is determined based on the number of recognition categories and the average precision corresponding to all recognition categories, and the mean average precision is determined as the recognition confidence level. This allows for the determination of whether to end the training of the behavior recognition model based on the recognition confidence level, thus ensuring the recognition accuracy of the behavior recognition model.

[0133] The above embodiments describe in detail the training method of the behavior recognition model provided in this application. In other embodiments, this application also provides a training apparatus for a behavior recognition model. Figure 8 This is a block diagram of a training apparatus for a behavior recognition model according to an embodiment of this application, such as... Figure 8As shown, the training device 700 for the behavior recognition model includes: a sample image acquisition module 710, used to acquire sample images of the driver and determine a training dataset based on the sample images, wherein the training dataset includes the actual behavior results of the sample images; a kernel weight determination module 720, used to input the training dataset into the behavior recognition model for recognition and determine the kernel weight of different target pixels in each sample image in the training dataset; a feature recombination determination module 730, used to reconstruct features in the behavior recognition model according to the kernel weight of different target pixels in each sample image to obtain an upsampled feature map; a recognition confidence determination module 740, used to perform behavior recognition on the upsampled feature map in the behavior recognition model to obtain a sample recognition result and determine a recognition confidence based on the sample recognition result and the actual behavior result; and a training module 750, used to determine a target loss function value based on the sample recognition result and the actual behavior result if the recognition confidence is less than or equal to a confidence threshold, update the behavior recognition model based on the target loss function value, and train the updated behavior recognition model.

[0134] In some embodiments, the training module 750 includes: a loss value determination submodule, configured to determine a first loss value and a second loss value based on the sample recognition result and the actual behavior result; a hyperparameter acquisition submodule, configured to acquire hyperparameters, wherein the hyperparameters are used to adjust the outlier sensitivity of the sample images in the training dataset; and a target loss function value determination submodule, configured to determine the target loss function value based on the hyperparameters, the first loss value, and the second loss value.

[0135] In some embodiments, the loss value determination submodule includes: an anchor frame determination unit, configured to determine the recognition anchor frame of the sample image based on the sample recognition result and the actual anchor frame of the sample image based on the actual behavior result; an intersection-union ratio (IUGR) determination unit, configured to determine the IUGR based on the recognition anchor frame and the actual anchor frame; and a first loss value determination unit, configured to determine the first loss value based on the IUGR, the recognition anchor frame, and the actual anchor frame.

[0136] In some embodiments, the first loss value determination unit includes: a center point acquisition subunit, configured to acquire a first center point of the identified anchor frame and a second center point of the actual anchor frame; a distance determination subunit, configured to determine a first target distance, a second target distance, and a Euclidean distance between the first center point and the second center point, wherein the first target distance is the distance between the first center point and the second center point on a first coordinate axis, and the second target distance is the distance between the first center point and the second center point on a second coordinate axis, and the first coordinate axis and the second coordinate axis are coordinate axes in different directions under the sample image coordinate system; and an angle cost determination subunit, configured to determine the angle cost based on the second target distance. The system comprises: a distance cost determination subunit, a distance cost determination subunit, a shape cost determination subunit, a shape cost determination subunit, a shape cost determination subunit, a first loss value ...

[0137] In some embodiments, the loss value determination submodule further includes: a diagonal determination unit, configured to determine the diagonal of the minimum bounding box of the identified anchor frame and the actual anchor frame; a target parameter determination unit, configured to determine a target parameter based on the diagonal and the Euclidean distance; and a second loss value determination unit, configured to determine the second loss value based on the target parameter.

[0138] In some embodiments, the target loss function value determination submodule includes: an intersection-over-union (IoU) mean determination unit, configured to determine the IoU mean based on the IoU corresponding to all sample images in the training dataset; an outlier coefficient determination unit, configured to determine the IoU corresponding to the current sample image and the IoU mean to determine the outlier coefficient; a target weight determination unit, configured to determine the target weight based on the outlier coefficient, the first hyperparameter, and the second hyperparameter; and a target loss function value determination unit, configured to determine the target loss function value based on the target weight, the first loss value, and the second loss value.

[0139] In some embodiments, the identification confidence determination module 740 includes: a first determination submodule, configured to determine the number of positive detections, the number of false positives, and the number of missed detections based on the sample identification results and the actual behavior results; a second determination submodule, configured to determine the accuracy based on the number of positive detections and the number of false positives, and to determine the recall based on the number of positive detections and the number of missed detections; a third determination submodule, configured to determine the average precision of each identification category identified in the sample identification results based on the recall; and an identification confidence determination submodule, configured to determine the mean average precision based on the number of identification categories and the average precision corresponding to all identification categories, and to determine the mean average precision as the identification confidence.

[0140] According to one aspect of the embodiments of this application, an electronic device is also provided, such as... Figure 9 As shown, the electronic device 800 also includes a processor 810 and one or more memories 820. The one or more memories 820 are used to store program instructions executed by the processor 810. When the processor 810 executes the program instructions, it implements the above-described training method for the behavior recognition model.

[0141] Furthermore, the processor 810 may include one or more processing cores. The processor 810 runs or executes instructions, programs, code sets, or instruction sets stored in the memory 820, and retrieves data stored in the memory 820. Optionally, the processor 810 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 810 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor and may be implemented using a separate communication chip.

[0142] According to one aspect of this application, a computer-readable storage medium is also provided, which may be included in the cloud server described in the above embodiments; or it may exist independently and not assembled into the cloud server. The aforementioned computer-readable storage medium carries computer-readable instructions that, when executed by a processor, implement the methods in any of the above embodiments.

[0143] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0144] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0145] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0146] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A training method for a behavior recognition model, characterized in that, The method includes: Acquire sample images of drivers and determine a training dataset based on the sample images, wherein the training dataset includes the actual behavioral results of the sample images; The training dataset is input into the behavior recognition model for recognition, and the kernel weights of different target pixels in each sample image in the training dataset are determined. Based on the kernel weights of different target pixels in each sample image, features are recombined in the behavior recognition model to obtain an upsampled feature map; In the behavior recognition model, behavior recognition is performed on the upsampled feature map to obtain sample recognition results, and the recognition confidence is determined based on the sample recognition results and the actual behavior results; If the recognition confidence is less than or equal to the confidence threshold, then the target loss function value is determined based on the sample recognition result and the actual behavior result, and the behavior recognition model is updated based on the target loss function value, and the updated behavior recognition model is trained.

2. The method according to claim 1, characterized in that, Determining the target loss function value based on the sample identification result includes: Determine the first loss value and the second loss value based on the sample identification results and the actual behavior results; Obtain hyperparameters, wherein the hyperparameters are used to adjust the outlier sensitivity of the sample images in the training dataset; The target loss function value is determined based on the hyperparameters, the first loss value, and the second loss value.

3. The method according to claim 2, characterized in that, The step of determining the first loss value and the second loss value based on the sample identification result and the actual behavior result includes: The recognition anchor box of the sample image is determined based on the sample recognition result, and the actual anchor box of the sample image is determined based on the actual behavior result; The intersection-union ratio is determined based on the identified anchor frame and the actual anchor frame; The first loss value is determined based on the cross-union ratio, the identified anchor frame, and the actual anchor frame.

4. The method according to claim 3, characterized in that, Determining the first loss value based on the intersection-union ratio, the identified anchor frame, and the actual anchor frame includes: Obtain the first center point of the identified anchor frame and the second center point of the actual anchor frame; Determine the first target distance, the second target distance, and the Euclidean distance between the first center point and the second center point, wherein the first target distance is the distance between the first center point and the second center point on the first coordinate axis, the second target distance is the distance between the first center point and the second center point on the second coordinate axis, and the first coordinate axis and the second coordinate axis are coordinate axes in different directions under the sample image coordinate system; The angular cost is determined based on the second target distance and the Euclidean distance. Obtain the first width and first height of the minimum bounding box between the identified anchor frame and the actual anchor frame, and determine the distance cost based on the first target distance, the second target distance, the angle cost, the first width, and the first height; Obtain the second width and second height of the identified anchor frame, the third width and third height of the actual anchor frame, and determine the shape cost based on the second width, second height, third width and third height; The first loss value is determined based on the distance cost, the shape cost, and the intersection-union ratio.

5. The method according to claim 3, characterized in that, The step of determining the first loss value and the second loss value based on the sample identification result and the actual behavior result includes: Determine the diagonal of the minimum bounding box in the identified anchor frame and the actual anchor frame; The target parameters are determined based on the diagonal and the Euclidean distance. The second loss value is determined based on the target parameters.

6. The method according to claim 3, characterized in that, The hyperparameters include a first hyperparameter and a second hyperparameter. Determining the target loss function value based on the hyperparameters, the first loss value, and the second loss value includes: The mean cross-union ratio is determined based on the cross-union ratios of all sample images in the training dataset. Determine the outlier coefficient by determining the intersection-union ratio (IU) of the current sample image and the mean IU; The target weight is determined based on the outlier coefficient, the first hyperparameter, and the second hyperparameter. The target loss function value is determined based on the target weight, the first loss value, and the second loss value.

7. The method according to any one of claims 1-6, characterized in that, The step of determining the recognition confidence level based on the sample recognition result and the actual behavior result includes: The number of positive detections, false detections, and missed detections are determined based on the sample identification results and the actual behavior results. Accuracy is determined based on the number of positive detections and the number of false positives, and recall is determined based on the number of positive detections and the number of missed detections. The average precision of each identified category in the sample identification results is determined based on the recall rate. The mean average precision is determined based on the number of categories of the identification category and the average precision corresponding to all identification categories, and the mean average precision is determined as the identification confidence level.

8. A training device for a behavior recognition model, characterized in that, The device includes: A sample image acquisition module is used to acquire sample images of the driver and determine a training dataset based on the sample images, wherein the training dataset includes the actual behavioral results of the sample images; The kernel weight determination module is used to input the training dataset into the behavior recognition model for recognition, and determine the kernel weight of different target pixels in each sample image in the training dataset; The feature recombination module is used to recombine features in the behavior recognition model according to the kernel weights of different target pixels in each sample image to obtain an upsampled feature map; The confidence module is used to perform behavior recognition on the upsampled feature map in the behavior recognition model, obtain sample recognition results, and determine the confidence level based on the sample recognition results and the actual behavior results. The training module is used to determine a target loss function value based on the sample recognition result and the actual behavior result if the recognition confidence is less than or equal to a confidence threshold, update the behavior recognition model based on the target loss function value, and train the updated behavior recognition model.

9. An electronic device, characterized in that, The electronic device includes: processor; A memory, wherein computer-readable instructions are stored thereon, which, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images