A visual identification method for lifting deflection of a dump body

Abstract

The present application belongs to the field of visual identification technology, and specifically relates to a visual identification method for lifting deflection of a self-unloading carriage body, comprising: collecting self-unloading vehicle lifting continuous video frames; obtaining time series difference entropy representing motion complexity; constructing a task preference vector reflecting functional bias according to the loss amplitude of the two auxiliary tasks after removing the unit; generating a prior mask from the preference vector, and extracting the activation norm by convolution with the feature map to obtain a structure sensitivity scalar representing spatial extraction capability; constructing a weighting function with the global time series difference entropy mean, and obtaining the unit comprehensive contribution degree by weighting and normalizing the three types of indicators; combining the comprehensive contribution degree with the preference vector distribution balance constraint to screen the optimal sub-network by the super-network solidification path, so as to build a carriage deflection identification model, and input the image to be tested to output the carriage lateral deflection angle. The present application can improve the accuracy and stability of the carriage lateral deflection angle identification.

Description

Technical Field

[0001] This invention relates to the field of visual recognition technology. More specifically, this invention relates to a visual recognition method for detecting tilting during the lifting of a dump truck body. Background Technology

[0002] During the material unloading and lifting operation of dump trucks, the truck body is prone to lateral tilting due to environmental factors such as uneven material distribution, poor ground flatness, asynchronous lifting of the hydraulic system, or interference from lateral wind. If this tilting is not detected and corrected in time, it can lead to reduced unloading efficiency and damage to the vehicle's mechanical structure, or even cause a rollover accident. Visual recognition-based tilt detection methods offer advantages such as non-contact measurement, rich information content, and low equipment modification costs. The lifting process of the dump truck body is a dynamic, time-series process, and construction sites often experience visual interference such as dust obstruction and drastic changes in lighting. This places high demands on the visual recognition model's ability to identify temporal motion features and extract spatial geometric features.

[0003] Currently, existing visual recognition methods, when dealing with the problem of tilting during dump truck lifting, typically rely on designing fixed deep neural network structures based on human experience or employing conventional neural architecture search techniques.

[0004] However, these network models cannot simultaneously account for temporal variations and spatial multi-scale feature representations within video frame sequences. Conventional network architecture search methods typically focus only on macroscopic, single performance metrics, lacking fine-grained evaluation of the actual contributions of each computational unit within the network to spatial feature extraction, temporal modeling, and collaborative processing of related auxiliary tasks. This results in network structures with unreasonable allocation of computational resources or redundant feature extraction, insufficient structural sensitivity to highly specialized tasks such as dump truck body tilting, and difficulty in matching the optimal network architecture to variations in the lifting process. Therefore, there is an urgent need for a method that can measure the comprehensive contribution of network computational units across multiple dimensions, including temporal features, spatial structure, and task preferences, and use this to drive network structure search, thereby constructing a visual recognition model to address the insufficient accuracy and reliability of dump truck body tilting recognition under actual working conditions. Summary of the Invention

[0005] To address the technical problem that existing methods lack fine-grained evaluation of the multidimensional contributions of computational units and are difficult to match the optimal network structure, this invention proposes a visual recognition method for the lifting and tilting of dump truck bodies, which can improve the accuracy and stability of recognizing the lateral tilt angle of the truck body.

[0006] In a first aspect, the present invention provides a visual recognition method for tilting during the lifting of a dump truck body, comprising: S1, acquiring a continuous video image frame sequence of the dump truck lifting process, and constructing a hypernetic search space containing candidate computational units; S2, for any candidate computational unit in the hypernetic search space, calculating the Shannon entropy of the difference matrix between adjacent frames based on the temporal feature map of the candidate computational unit, and obtaining the temporal difference entropy representing the motion complexity; S3, constructing a task preference vector representing functional bias based on the increase in loss function caused by removing candidate computational units in corner regression auxiliary tasks and pixel segmentation auxiliary tasks; S4, generating a prior mask based on the task preference vector, and combining the prior mask with the image feature map. After performing convolution, the norm of the activation response matrix is ​​extracted to obtain a structural sensitivity scalar representing the spatial feature extraction capability; S5, using a weighting function with the mean of the global temporal difference entropy as the independent variable, the normalized temporal difference entropy, the structural sensitivity scalar, and the magnitude of the task preference vector are combined to obtain the comprehensive contribution of the candidate computing units; S6, based on the comprehensive contribution and the balance constraint on the distribution of the task preference vector, a fixed path is selected in the hypernetwork search space to determine the optimal subnetwork structure; S7, a skew recognition model is constructed using the optimal subnetwork structure, the image to be tested is input into the skew recognition model for processing, and the lateral skew angle of the dump truck body is output.

[0007] By adopting the above technical solution, calculating the temporal difference entropy, constructing the task preference vector, extracting the structural sensitivity scalar, and combining global weighting and balance constraints to comprehensively evaluate the contribution of each candidate computing unit, thereby driving the hypernetwork search to determine the optimal subnetwork structure, the accuracy and stability of visual recognition of the lateral tilt angle of dump truck bodies can be improved.

[0008] Preferably, the step of acquiring a continuous video image frame sequence of the dump truck lifting process and constructing a hypernetwork search space containing candidate computing units includes: using a residual network with a preset depth as the backbone network architecture, and setting multiple convolutional kernels and skip connections of different receptive field sizes in parallel at each level node of the backbone network architecture as candidate computing units; constructing all possible directed connection combinations between each level node to obtain a hypernetwork search space covering multiple receptive field scales and feature fusion paths.

[0009] By adopting the above technical solution, a residual network with a preset depth is used as the backbone architecture. Multiple convolutional kernels with different receptive fields and skip connections are set in parallel at each level node, and a fully directed connection combination between each level node is constructed. This can cover multiple receptive field scales and feature fusion paths, providing a rich parameterized optimization space for subsequent architecture search.

[0010] Preferably, the step of calculating the Shannon entropy of the difference matrix between adjacent frames based on the temporal feature map of the candidate computing unit to obtain the temporal difference entropy representing the motion complexity includes: acquiring the feature maps of two adjacent frames output by the continuous video image frame sequence when passing through the candidate computing unit; performing an absolute difference operation on the two frame feature maps to obtain a difference matrix; performing hard thresholding denoising on the difference matrix using a preset threshold to remove noise elements with absolute values ​​less than the threshold and retaining non-zero elements representing effective motion regions; dividing the numerical distribution of non-zero elements into a preset number of intervals and calculating the probability ratio of the number of non-zero elements in each interval to the total number of non-zero elements; and calculating the information entropy based on each probability ratio as the temporal difference entropy of the candidate computing unit.

[0011] By adopting the above technical solution, hard thresholding is performed on the difference matrix of the feature maps of two adjacent frames to retain the non-zero elements of the effective motion region. The amplitude distribution is divided into intervals and the information entropy is calculated as the temporal difference entropy. This can effectively characterize the temporal perception ability of each candidate computing unit of the motion complexity during the lifting process and suppress background noise interference.

[0012] Preferably, the step of constructing a task preference vector representing functional bias based on the increase in loss function caused by removing candidate computation units in the corner regression auxiliary task and the pixel segmentation auxiliary task includes: calculating the relative increase in the first loss function of the corner regression auxiliary task and the relative increase in the second loss function of the pixel segmentation auxiliary task respectively when removing candidate computation units compared to when candidate computation units are included, wherein the relative increase is the ratio of the absolute loss increase to the baseline loss value of the corresponding auxiliary task, and setting the relative increase less than zero to zero; and constructing the task preference vector based on the relative increase in the first loss function and the relative increase in the second loss function.

[0013] By adopting the above technical solution, a task preference vector is constructed by removing the relative increase in the loss values ​​of the two auxiliary tasks before and after the candidate computing unit, and the relative increase that is less than zero is set to zero. This can eliminate the difference in physical dimensions and ensure the non-negative monotonicity of the contribution index, accurately reflecting the actual impact of the unit on multi-task performance.

[0014] Preferably, the step of constructing a task preference vector based on the relative increase of the first loss function and the relative increase of the second loss function includes: constructing a two-dimensional vector in a preset two-dimensional task coordinate system with the relative increase of the first loss function as the first coordinate value and the relative increase of the second loss function as the second coordinate value; using the two-dimensional vector as the task preference vector, where the magnitude represents the overall contribution of the corner point regression auxiliary task and the pixel segmentation auxiliary task, and the angle between the vector and the coordinate axis of the two-dimensional task coordinate system represents the functional bias towards fine positioning or region segmentation.

[0015] By adopting the above technical solution, the relative increase of the two losses is used as the coordinate values ​​of the two-dimensional task coordinate system to construct a two-dimensional vector. The vector magnitude represents the overall contribution across tasks, and the angle with the coordinate axis represents the functional bias for fine localization or region segmentation. This can intuitively characterize the task function tendency of each candidate computing unit and provide a basis for the functional balance design of the network structure.

[0016] Preferably, the step of generating a priori mask based on the task preference vector includes: extracting the angle between the task preference vector and the coordinate axis corresponding to the corner regression auxiliary task; when the angle is less than a preset angle threshold, generating a two-dimensional high-pass filter with a preset spatial size that focuses on enhancing high-frequency contour edge features as a priori mask; when the angle is greater than or equal to the preset angle threshold, generating a two-dimensional low-pass filter with a preset spatial size that focuses on smoothly extracting low-frequency flat region features as a priori mask.

[0017] By adopting the above technical solution, the angle between the task preference vector and the corner regression task coordinate axis is extracted. High-pass or low-pass prior masks are generated according to whether the angle is less than or greater than or equal to the angle threshold. The spatial feature enhancement strategy can be adaptively selected according to the functional bias of the unit, thereby improving the structural perception accuracy of rigid edges or flat areas.

[0018] Preferably, the step of extracting the activation response matrix norm after convolving the prior mask with the image feature map to obtain a structural sensitivity scalar representing the spatial feature extraction capability includes: acquiring the spatial feature map generated by the candidate computing unit on a single-frame search validation set image; performing a sliding convolution operation on the prior mask and the spatial feature map in the spatial dimension to generate an activation response matrix representing the degree of corresponding feature enhancement; calculating the L2 norm of the activation response matrix, and normalizing the calculated L2 norm result as the structural sensitivity scalar.

[0019] By adopting the above technical solution, the prior mask and the spatial feature map of the candidate computing unit on the validation set image are subjected to sliding convolution, and the L2 norm of the activation response matrix is ​​calculated and normalized as a structural sensitivity scalar. This can objectively characterize the ability of each unit to extract and enhance different spatial structures, and make up for the insufficiency of single task indicators in measuring spatial expressiveness.

[0020] Preferably, the step of using a weighting function with the mean of the global temporal differential entropy as the independent variable to combine the normalized temporal differential entropy, the structural sensitivity scalar, and the magnitude of the task preference vector to obtain the comprehensive contribution of the candidate computing units includes: calculating the average of the temporal differential entropy of all candidate computing units as the mean of the global temporal differential entropy; constructing a weighting function containing a first weight coefficient and a second weight coefficient, wherein the first weight coefficient is positively correlated with the mean of the global temporal differential entropy, and the second weight coefficient is negatively correlated with the mean of the global temporal differential entropy and is subject to a non-negative lower bound constraint; and using the weighting function to combine the normalized temporal differential entropy, the structural sensitivity scalar, and the magnitude of the task preference vector to obtain the comprehensive contribution of the candidate computing units.

[0021] By adopting the above technical solution, the mean of the temporal differential entropy of all candidate computing units is used as the global parameter to construct a first weight coefficient that is positively correlated with it and a second weight coefficient with a non-negative lower bound constraint, thereby forming an adaptive weighting function that can dynamically balance the weight ratio of temporal features and spatial structure features in the comprehensive evaluation according to the change of overall motion complexity.

[0022] Preferably, the step of combining the normalized temporal difference entropy, structural sensitivity scalar, and task preference vector magnitudes using a weighting function to obtain the comprehensive contribution of the candidate computing unit includes: weighting the normalized temporal difference entropy using a first weighting coefficient, weighting the structural sensitivity scalar using a second weighting coefficient, and nonlinearly summing the weights with the normalized task preference vector magnitudes to obtain the comprehensive contribution.

[0023] By adopting the above technical solution, the first weight coefficient is used to weight the normalized temporal difference entropy, the second weight coefficient is used to weight the structural sensitivity scalar, and the normalized task preference vector magnitude is nonlinearly summed. This can effectively integrate three different types of evaluation indicators into a single comprehensive contribution scalar, taking into account temporal sensitivity, spatial expressiveness and multi-task collaborative ability.

[0024] Preferably, the step of selecting a fixed path in the hypernetwork search space to determine the optimal subnetwork structure based on the comprehensive contribution and the balance constraint of the task preference vector distribution includes: in each round of the search iteration, removing candidate computing units and related connection paths whose comprehensive contribution is lower than a preset threshold; statistically analyzing the angle distribution of the task preference vectors of the retained candidate computing units; if the proportion of units focusing on fine positioning or region segmentation exceeds a preset balance ratio range, penalizing the comprehensive contribution of the majority of units and restoring the connection paths of the removed minority units; when the network structure converges or reaches the maximum number of iterations, fixing the retained candidate computing units and connection paths to obtain the optimal subnetwork structure.

[0025] By adopting the above technical solution, low-contribution units are removed in each search iteration, the functional bias distribution of retained units is statistically analyzed, penalties are imposed on majority units that exceed the equilibrium ratio range, and connection paths of minority units are restored. This allows for maintaining a dynamic balance between fine localization and region segmentation functions during the pruning optimization process, resulting in an optimal subnetwork with a simplified structure and complementary functions.

[0026] The present invention has the following beneficial effects:

[0027] This invention utilizes the Shannon entropy of the difference matrix between adjacent frames to obtain temporal difference entropy, characterizing the motion complexity during the lifting process of a dump truck. It constructs a task preference vector by using the loss function increment of corner regression and pixel segmentation to assist the task. Combined with the structural sensitivity scalar extracted based on prior masks and image feature map convolution, a weighting function with the mean of the global temporal difference entropy as the independent variable is used to combine the normalized temporal difference entropy, structural sensitivity scalar, and task preference vector magnitude to evaluate the comprehensive contribution of each candidate computational unit. Furthermore, by incorporating balance constraints to lock the optimal sub-network structure, a skew recognition model with both a streamlined architecture and excellent feature expressive power is constructed, improving the accuracy and stability of visual recognition of the lateral skew angle of the dump truck body in various scenarios.

[0028] Furthermore, this invention constructs a hypernetwork search space encompassing multiple receptive field scales and feature fusion paths, and employs a residual network with a preset depth as the backbone architecture, providing a rich foundation for parameterized optimization of network structure search. By performing hard thresholding denoising on the difference matrix and calculating temporal difference entropy based on the probability distribution of non-zero element amplitudes, it effectively suppresses the interference of background noise on motion feature extraction. By mapping the relative increases of the two losses to a two-dimensional task coordinate system and representing the overall contribution and functional bias with vector magnitude and included angle, respectively, it achieves an intuitive characterization of the task functional tendency of each candidate computational unit. Based on... Functionally biased adaptive generation of high-pass or low-pass prior masks and calculation of the L2 norm of the activation response matrix as a structural sensitivity scalar objectively characterize the ability of each unit to extract and enhance different spatial structures. By dynamically adjusting the weight ratio of temporal features and spatial structural features with the mean of global temporal differential entropy as the independent variable, and fusing multiple evaluation indicators in a nonlinear summation manner, a reasonable integration of indicators with different dimensions is achieved. By maintaining a dynamic balance between fine localization and region segmentation functions during search iteration, the functional complementarity of the network structure during pruning optimization is ensured, further improving the stability of the skew identification model under complex conditions. Attached Figure Description

[0029] Figure 1 This is a flowchart of a visual recognition method for the lifting and tilting of a dump truck body; Figure 2 It is a discrete probability histogram of the amplitude distribution of non-zero elements in a continuous motion region; Figure 3 This is a scatter plot of the task preference vector distribution of each candidate computational unit in the hypernetwork. Detailed Implementation

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0031] This invention discloses a visual recognition method for detecting tilting during the lifting of a dump truck body, referring to... Figure 1 This includes steps S1-S7: S1. Obtain a continuous video image frame sequence of the dump truck lifting process and construct a hypernet search space containing candidate computing units.

[0032] Load the dump truck lifting video, read the image data frame by frame, scale the resolution of each frame to a fixed size, and convert it into tensor format to form a continuous video image frame sequence.

[0033] In one embodiment, a set of operations including 3×3 convolution, 5×5 convolution, max pooling, average pooling, and depthwise separable convolution is defined as candidate computational units. These operation sets are then connected as nodes in a directed acyclic graph to construct the hypernetwork search space.

[0034] In an optional embodiment, a continuous video image frame sequence of the dump truck lifting process is obtained, and a hypernetwork search space containing candidate computing units is constructed, including: using a residual network with a preset depth as the backbone network architecture, setting multiple convolutional kernels and skip connections of different receptive field sizes in parallel at each level node of the backbone network architecture as candidate computing units; constructing all possible directed connection combinations between each level node to obtain a hypernetwork search space covering multiple receptive field scales and feature fusion paths.

[0035] In this embodiment, the hypernetwork model takes a continuous sequence of video image frames from a dump truck as input and outputs a high-dimensional feature map encompassing multi-scale feature fusion, providing a parameterized optimization space for the subsequent architecture search stage. The hypernetwork model structure includes four hierarchical nodes based on a 50-layer residual network. Each hierarchical node contains several residual microstructure units based on dense connections of directed acyclic graphs. At the feature processing nodes of the microstructure units, seven candidate computational unit combinations are arranged side-by-side.

[0036] Using a 50-layer residual network with a preset depth as the basic prototype of the backbone network architecture, the network is divided into four levels of nodes according to the downsampling rules of spatial resolution. Each level of node contains several residual microstructure units.

[0037] When constructing candidate computational units, seven basic operators with different receptive fields or computational properties are arranged side-by-side at the feature processing nodes of each microstructural unit. These include standard two-dimensional convolutional kernels with sizes of 3×3, 5×5, and 7×7, a 3×3 dilated convolution with a dilation rate of 2, and 3×3 max-pooling and average-pooling layers for spatial feature dimensionality reduction. Additionally, identity skip connections are utilized to preserve the transmission of low-level detailed features. These candidate computational units collectively carry the network weight parameters of the supernetwork and the architectural weight parameters representing the importance of operators, and are alternately subjected to gradient optimization during the search phase.

[0038] A directed acyclic graph topological logic is used to densely connect the input and output nodes within each microstructure unit. For the j-th feature fusion computing node within the i-th level node, the feature input not only comes from the output of the (j-1)-th predecessor node, but can also establish cross-level directed connections with the (j-2)-th or even earlier microstructure nodes.

[0039] The output characteristic of the j-th node within the microstructural unit can be expressed as: ,in Let the output feature map tensor of the k-th predecessor node be . This is a hybrid operation operator that includes combinations of the above seven candidate computational units.

[0040] On each established directed feature transfer connection edge, the seven candidate computation unit combination branches set up in parallel are placed without difference. In this way, a parameterized network optimization space is formed by arranging and combining these branches, which covers local fine-grained feature extraction, large-scale structural context awareness, and original image resolution feature fusion path. This ensures that subsequent algorithms can select the optimal fixed path for the carriage posture recognition task under a large degree of structural freedom.

[0041] S2, for any candidate computational unit in the hypernet search space, calculate the Shannon entropy of the difference matrix between adjacent frames based on the temporal feature map of the candidate computational unit, and obtain the temporal difference entropy representing the motion complexity.

[0042] Extract the temporal feature map tensors output by the candidate computation unit after inputting the current frame and the previous frame image into the candidate computation unit, subtract them element by element, and take the absolute value to generate a difference matrix.

[0043] The pixel values ​​of the difference matrix are converted into a probability distribution. The base-2 logarithm of each probability is calculated. The probability distribution is multiplied element by element by its logarithm, the sum is taken and the opposite is taken, and the temporal difference entropy representing the motion complexity of the local region is calculated.

[0044] In an optional embodiment, the Shannon entropy of the difference matrix between adjacent frames is calculated based on the temporal feature map of the candidate computing unit to obtain the temporal difference entropy representing the motion complexity. This includes: acquiring the feature maps of two adjacent frames output by the continuous video image frame sequence when passing through the candidate computing unit; performing an absolute difference operation on the two frame feature maps to obtain a difference matrix; performing hard thresholding denoising on the difference matrix using a preset threshold to remove noise elements with absolute values ​​less than the threshold and retaining non-zero elements representing effective motion regions; dividing the numerical distribution of non-zero elements into a preset number of intervals and calculating the probability ratio of the number of non-zero elements in each interval to the total number of non-zero elements; and calculating the information entropy based on each probability ratio as the temporal difference entropy of the candidate computing unit.

[0045] During inference, two consecutive frames of original video images input at time t and time t-1 during the dump truck lifting process are extracted. After these frames pass through specific candidate computation units at the current level, two feature map tensors of size denoted as channel number multiplied by height multiplied by width are generated. A pixel-wise absolute difference operation is performed on these two spatially scale-aligned feature map tensors, and mean compression is calculated along the channel dimension to reduce computational complexity, resulting in a single-channel temporal difference matrix with a spatial resolution still equal to height multiplied by width.

[0046] Set a hard filtering threshold, preferably between 0.03 and 0.08, for example, 0.05, to filter out background static noise caused by hydraulic rod vibration, sudden changes in outdoor ambient light, and video stream encoding / decoding compression. Use this threshold to perform hard thresholding denoising on the single-channel temporal difference matrix, that is, truncate response elements with absolute values ​​less than 0.05 in the matrix to zero, thereby separating and retaining the effective non-zero pixel group that can represent the continuous lateral tilting motion of the dump truck body.

[0047] The amplitude data of non-zero elements in the effective motion region after filtering are extracted. Given that the preceding feature map has been truncated using an activation function, the non-zero amplitudes are distributed within a bounded real number range of 0.05 to 1. This bounded numerical distribution range is divided into a preset number of 20 discrete statistical histogram intervals with equal numerical step sizes. Preferably, the number of intervals is set to 20, i.e., the step size of each interval is 0.0475.

[0048] By iterating through the group of non-zero elements, counting the number of elements falling within these 20 discrete intervals using a counter, and dividing each number by the total number of non-zero elements in the matrix, a discrete probability distribution ratio sequence representing the frequency proportion of each amplitude is obtained. to The discrete probability distribution of the amplitudes of the non-zero elements in the effective motion region described above can be presented using a histogram, such as... Figure 2 As shown in the figure, the effective non-zero element amplitude is plotted on the horizontal axis, and the probability ratio of non-zero elements within the interval is plotted on the vertical axis, representing the overall amplitude distribution characteristics of the lifting motion region of the dump truck body.

[0049] Substituting this probability ratio sequence into the classic Shannon entropy formula, we can calculate the probability ratio p for each interval. i and The calculated scalar information entropy is assigned as the temporal differential entropy of the specific candidate computational unit. The larger the entropy value, the more sensitive the operator is to the recognition and representation of multimodal temporal changes in the lifting car.

[0050] S3. Based on the increase in loss function caused by removing candidate computation units in corner regression auxiliary task and pixel segmentation auxiliary task, construct a task preference vector representing functional bias.

[0051] A leave-one-out strategy is adopted. The SmoothL1Loss algorithm is used to calculate the corner regression loss value when the candidate computation unit is included in the supernetwork, and the CrossEntropyLoss algorithm is used to calculate the pixel segmentation loss value when the unit is included. The weight of the candidate computation unit is temporarily set to zero to isolate it from the forward propagation computation graph, and the corner regression loss value and pixel segmentation loss value after removing the unit are calculated again.

[0052] Subtracting the corner regression loss value before inclusion from the corner regression loss value after removal, and then dividing by the baseline corner regression loss value before inclusion, yields the relative increase in the corner regression auxiliary task loss function. Subtracting the pixel segmentation loss value before inclusion from the pixel segmentation loss value after removal, and then dividing by the baseline pixel segmentation loss value before inclusion, yields the relative increase in the pixel segmentation auxiliary task loss function. Concatenating these two relative increases on the same numerical scale generates a two-dimensional task preference vector, representing the functional bias of this unit.

[0053] In an optional embodiment, a task preference vector representing functional bias is constructed based on the increase in loss function caused by removing candidate computation units in the corner regression auxiliary task and the pixel segmentation auxiliary task. This includes: calculating the relative increase in the first loss function of the corner regression auxiliary task and the relative increase in the second loss function of the pixel segmentation auxiliary task when removing candidate computation units compared to when candidate computation units are included, respectively. The relative increase is the ratio of the absolute loss increase to the baseline loss value of the corresponding auxiliary task, and relative increases less than zero are set to zero; constructing a two-dimensional vector in a preset two-dimensional task coordinate system using the relative increase in the first loss function as the first coordinate value and the relative increase in the second loss function as the second coordinate value; and using the two-dimensional vector as the task preference vector, where the magnitude represents the overall contribution to the corner regression auxiliary task and the pixel segmentation auxiliary task, and the angle between the vector and the coordinate axes of the two-dimensional task coordinate system represents the functional bias towards fine localization or region segmentation.

[0054] During the intermediate iterations of the search phase, when the full super network structure is included, the baseline loss value of the corner regression auxiliary task obtained by the network forward inference for the current image batch is calculated, such as the smoothing L1 loss of the corner of the carriage and the baseline loss value of the pixel segmentation auxiliary task, such as the binary cross-entropy loss of the carriage skin region.

[0055] Feature masking is applied to a specific candidate computational unit in the supernetwork, specifically by using a zero-multiplier operator to short-circuit its output feature activation tensor, artificially representing the missing scenario removed by this operator in the network architecture. In this incomplete, missing network state, the forward propagation process is re-executed, and new loss values ​​for the two auxiliary tasks are calculated. The new corner regression loss is subtracted from the baseline value and then divided by the baseline value to eliminate the difference in physical dimensions, yielding the relative increase of the first loss function. The new pixel segmentation loss is subtracted from the baseline value and then divided by the baseline value, yielding the relative increase of the second loss function.

[0056] If the new loss decreases instead of increasing, it indicates that the operator may introduce an incorrect gradient. Therefore, by using activation constraints of the linear rectified function type, any relative increase in loss less than zero is set to zero to ensure that the contribution index conforms to the non-negative monotonicity principle.

[0057] In the pre-defined mathematical model, a two-dimensional Cartesian task coordinate system specifically used for operator capability projection is employed. In this space, the real value of the relative increase of the first loss function is used as the x-axis in the positive direction, and the real value of the relative increase of the second loss function is used as the y-axis in the positive direction. Since the coordinate values ​​of both axes have been converted into dimensionless relative rates of change, a two-dimensional bias feature vector is constructed from the origin to the endpoint coordinates X and Y, which is the task preference vector of this unit.

[0058] The Euclidean distance magnitude of the task preference vector is calculated as the square root of the sum of the squares of X and Y. This scalar magnitude represents the overall cross-task contribution of the unit to improving positioning accuracy and regional integrity. Optionally, if the relative increases of the first and second loss functions are both set to zero, resulting in a zero task preference vector, the subsequent angle calculation and prior mask extraction steps are skipped, and the structural sensitivity scalar and overall contribution of the candidate computational unit are forcibly set to zero. Simultaneously, the angle between the vector and the X-axis is calculated using inverse trigonometric functions. The angle represents the operator's functional design bias. When the angle approaches zero degrees, it indicates that its effect is concentrated on the fine positioning features of the vehicle's edge lines and corners. When the angle approaches 90°, it indicates that the computational unit exhibits a performance bias in extracting and retaining large-scale color blocks and structural surface region segmentation information of the vehicle body. The distribution of the task preference vectors of all candidate computational units in the hypernet in the two-dimensional task coordinate system is as follows: Figure 3 As shown in the figure, the scatter plot uses the first increase in loss of the corner regression auxiliary task as the horizontal axis and the second increase in loss of the pixel segmentation auxiliary task as the vertical axis. Each scatter point corresponds to the functional bias coordinate of a candidate computational unit.

[0059] S4. Generate a priori mask based on the task preference vector. After convolving the priori mask with the image feature map, extract the norm of the activation response matrix to obtain a structural sensitivity scalar that represents the spatial feature extraction capability.

[0060] A multilayer perceptron model is constructed to upscale the two-dimensional task preference vector to a vector with the same number of channels as the image feature map. The vector element values ​​are normalized to the range of 0 to 1 using the sigmoid activation function. The upscaled vector is then expanded in space and copied to a size with the same resolution as the image feature map to generate a priori mask.

[0061] The prior mask is multiplied element-wise with the image feature map output from the forward propagation of the candidate computation unit to characterize the mask convolution process. The result is processed by the ReLU activation function to obtain the activation response matrix. The F-norm of the activation response matrix is ​​calculated and used as a structural sensitivity scalar representing the spatial feature extraction capability of the unit.

[0062] In an optional embodiment, generating a prior mask based on the task preference vector includes: extracting the angle between the task preference vector and the coordinate axis corresponding to the corner regression auxiliary task; when the angle is less than a preset angle threshold, generating a two-dimensional high-pass filter with a preset spatial size that focuses on enhancing high-frequency contour edge features as a prior mask; when the angle is greater than or equal to the preset angle threshold, generating a two-dimensional low-pass filter with a preset spatial size that focuses on smoothly extracting low-frequency flat region features as a prior mask.

[0063] The key geometric attributes representing the operator features are extracted from the constructed task preference vector data structure. Specifically, the absolute counterclockwise angle between this two-dimensional vector and the baseline coordinate axis representing the corner regression auxiliary task direction is extracted, with the effective data range defined between 0° and 90°. Internally, 45° is fixed as a preset angle decision threshold for classifying the visual preference attributes of the operator.

[0064] During the logical judgment execution process, when the extraction angle of a certain calculation unit is read to be less than a preset angle threshold of 45°, it is determined that its function is biased towards parsing the rigid border and four corners of the dump truck body. A Laplacian edge detection operator with a preset spatial physical size of 3×3 or a Sobel convolution kernel based on horizontal and vertical bidirectional gradients is generated as a priori mask matrix. The weight matrix of this type of high-pass filter has spatial suppression characteristics with strong central response and negative peripheral weights, which can stimulate rigid edges with abrupt gray-level changes on the feature map and amplify high-frequency contour information.

[0065] Correspondingly, when the angle read is greater than or equal to the 45° angle threshold, it is determined that the main function of the current candidate computational unit is to construct continuous features for the large-area material region of the dump truck's side baffle and chassis inner wall. For these feature regions that are easily affected by point noise, a 5×5 Gaussian filter kernel with an appropriately enlarged spatial size and a preset standard deviation parameter of 1.2 is generated, or a 5×5 mean filter matrix with identical weights is used as a second-type prior mask.

[0066] In this weight matrix with two-dimensional low-pass filtering characteristics, the element weights decrease exponentially from the kernel center to the boundary, and the sum of all elements is normalized to 1. This mechanism can filter out high-frequency outliers caused by camera equipment contamination, allowing for priority emphasis and complete preservation of low-frequency, flat, large-area features with uniform distribution and gentle gradient transformation during convolution enhancement, thereby improving the accuracy of feature representation evaluation.

[0067] In an optional embodiment, the activation response matrix norm is extracted after convolving the prior mask with the image feature map to obtain a structural sensitivity scalar representing the spatial feature extraction capability. This includes: obtaining the spatial feature map generated by the candidate computing unit on a single-frame search validation set image; performing a sliding convolution operation on the prior mask and the spatial feature map in the spatial dimension to generate an activation response matrix representing the degree of corresponding feature enhancement; calculating the L2 norm of the activation response matrix; and normalizing the calculated L2 norm result as the structural sensitivity scalar.

[0068] During the architecture search and verification of the supernetwork, after forward propagation of a frame of dump truck lifting search and verification set image randomly sent to the system, the current specific candidate computation unit is read and extracted from the specified level cache of the backbone network, such as the spatial feature map of the three-dimensional tensor generated by the forward computation of a 5×5 ordinary convolutional layer.

[0069] If the feature map has a depth of multiple channels, such as 64 channels, the multi-channel feature map is averaged and reduced along the channel stacking dimension to generate a two-dimensional single-channel high-dimensional abstract feature map plane that represents the original spatial geometric mapping information.

[0070] Based on the pre-prepared prior mask matrix (such as a 3×3 Laplacian mask or a 5×5 Gaussian smooth mask used in the preceding decision), this single-channel plane is used as the target input matrix. A zero-padding strategy with a stride of 1 is configured to ensure boundary scale alignment. The prior mask then performs pixel-wise sliding inner product convolution operations from left to right and from top to bottom across the entire feature map's spatial dimensions. The activation values ​​can be recombined using the unique weight layout within the mask to generate an activation response matrix that represents the network's sensitivity to certain spatial detail or structural enhancement mappings.

[0071] The F-norm is calculated for all pixels in the generated activation response matrix. The specific steps are: square all activation values ​​in the matrix, sum these squares globally, and then take the square root of the sum. Since the feature energy base extracted by operators in shallow and deep layers of the network differs significantly, applying the absolute F-norm is not reasonable. Therefore, a minimax feature scaling algorithm is used to find the maximum and minimum values ​​of this norm set within all parallel candidate operators in the current network.

[0072] By subtracting the minimum value from the original F-norm and dividing by the difference between the maximum and minimum values, this descriptor value is linearly projected and compressed into a standard closed interval of 0 to 1.

[0073] The floating-point number between 0 and 1 obtained after the above normalization correction is used as an indicator of the ability of the current candidate computing unit to extract and retain key visual features, namely the structural sensitivity scalar.

[0074] S5 uses a weighting function with the mean of the global temporal difference entropy as the independent variable to combine the normalized temporal difference entropy, the structural sensitivity scalar, and the magnitude of the task preference vector to obtain the comprehensive contribution of the candidate computing unit.

[0075] The MinMaxScaler normalization algorithm is used to linearly scale the temporal difference entropy, structural sensitivity scalar, and task preference vector calculated based on Euclidean distance of all candidate computing units to the real number interval of 0 to 1.

[0076] Calculate the arithmetic mean of the temporal difference entropy of all candidate computational units within the current layer, and use it as the global temporal difference entropy mean. Input the global temporal difference entropy mean into a weighting function constructed based on the Softmax function to calculate three weight coefficients corresponding to the temporal difference entropy, the structural sensitivity scalar, and the task preference vector magnitude, respectively.

[0077] The three weighting coefficients are multiplied by their corresponding normalized indices, and the sum is obtained by addition to obtain the overall contribution of the candidate computing unit.

[0078] In an optional embodiment, a weighting function with the mean of the global temporal differential entropy as the independent variable is used to combine the normalized temporal differential entropy, the structural sensitivity scalar, and the magnitude of the task preference vector to obtain the comprehensive contribution of the candidate computing units. This includes: calculating the average of the temporal differential entropy of all candidate computing units as the mean of the global temporal differential entropy; constructing a weighting function containing a first weighting coefficient and a second weighting coefficient, wherein the first weighting coefficient is positively correlated with the mean of the global temporal differential entropy, the second weighting coefficient is negatively correlated with the mean of the global temporal differential entropy and is subject to a non-negative lower bound constraint; weighting the normalized temporal differential entropy with the first weighting coefficient, weighting the structural sensitivity scalar with the second weighting coefficient, and nonlinearly summing the result with the magnitude of the normalized task preference vector to obtain the comprehensive contribution.

[0079] Perform a system state review in the current iteration round. Collect the temporal difference entropy produced by all candidate computational units at each branch level node in the full hypernetic search space within the current period, and calculate the global temporal difference entropy mean, which represents the complex characteristics of the carriage motion during this period, by dividing the sum of all these values ​​by the total number of candidate units in the search space.

[0080] Based on these real-time global parameters, a weighted function system for responding to environmental complexity is configured using a hyperbolic tangent nonlinear smoothing function. The first weight coefficient, used to control the proportion of motion temporal evaluation, is defined as the global temporal differential entropy mean multiplied by a preset scaling gain, then taking an S-shaped function or tangent response, ensuring that this coefficient is positively correlated with increasing motion complexity.

[0081] Meanwhile, the second weighting coefficient, which dominates the analysis of a single frame, is calculated as 1 minus the first weighting coefficient. To ensure that the evaluation of spatial static structure does not fail even in specific detection frames where the vehicle body is stationary (i.e., the differential entropy is close to zero), a non-negative lower bound constraint threshold of no less than 0.25 is set mathematically for the second weighting coefficient.

[0082] During the comprehensive capability integration phase, the temporal difference entropy values ​​produced by each individual computing unit and the extracted and calculated structural sensitivity scalars, along with the magnitude values ​​of the task preference vectors calculated by the Pythagorean theorem for the horizontal and vertical coordinates, are transformed using the minimax method or normalized exponential function to flatten the distribution to the same baseline scale of 0 to 1.

[0083] The evaluation is performed by operator superposition according to the fusion logic. The first weighting coefficient is directly multiplied by the normalized individual temporal difference entropy to obtain the temporal feature sensitivity score assigned to the current stage. Then, the second weighting coefficient is multiplied by the individual's structural sensitivity scalar to obtain the spatial structure recognition score.

[0084] By incorporating nonlinear transformations with natural logarithms or squared terms, such as a mapping structure using 1 plus the natural logarithm of the normalized task preference vector magnitude, multi-task preference support performance is integrated. The three sets of processed data values ​​are then scalar-level algebraically summed and merged. The resulting total score is a scalar value representing the overall contribution.

[0085] S6. Based on the comprehensive contribution and the balance constraint on the distribution of task preference vectors, a fixed path is selected in the hypernetwork search space to determine the optimal subnetwork structure.

[0086] Using the KL divergence algorithm, the relative entropy divergence value between the task preference vector distribution of each candidate computing unit and the preset uniform distribution is calculated, and this value is used as a balance constraint. The fitness score of each candidate computing unit is calculated by subtracting the balance constraint multiplied by the preset regularization coefficient from the overall contribution.

[0087] The Viterbi programming algorithm is used to search for the sequence of operation nodes with the largest sum of global fitness scores in the directed acyclic graph formed by the hypernetwork search space, and this sequence is selected as the fixed path. Unselected computational branches outside the fixed path are removed, and the remaining network topology is determined as the optimal subnetwork structure.

[0088] In an optional embodiment, the optimal sub-network structure is determined by selecting a fixed path in the hypernetwork search space based on the overall contribution and the balance constraint on the distribution of task preference vectors. This includes: in each round of the search iteration, removing candidate computing units and related connection paths whose overall contribution is lower than a preset threshold; statistically analyzing the angle distribution of the task preference vectors of the retained candidate computing units; if the proportion of units focusing on fine-grained localization or region segmentation exceeds a preset balance ratio range, penalizing the overall contribution of the majority of units and restoring the connection paths of the removed minority units; when the network structure converges or reaches the maximum number of iterations, the retained candidate computing units and connection paths are fixed to obtain the optimal sub-network structure.

[0089] In the gradient-based hypernetwork architecture and weight-alternating optimization parameter search process, at the end of each network training iteration, the pruning algorithm begins to perform a traversal scan of the entire space. For each candidate computational unit at each computational level, the algorithm extracts the comprehensive contribution scalar of its previous stage and compares it with a preset survival threshold, preferably set to 0.35. When the scalar value is lower than the 0.35 threshold, this operator is determined to be a redundant component for skew attitude detection and is removed from the directed acyclic graph of the hypernetwork's internal connection computation flow. The corresponding directed connection edges for tensor feedforward and backpropagation error propagation are also blocked and not processed, thereby achieving a gradual reduction in the hypernetwork's memory size across generations and making the retained paths more focused.

[0090] For the computational units retained in this round, the degree distribution results of the angle between their respective task preference vectors are read from each retained unit for a second determination. That is, the total number of computational units with an angle less than 45° that are determined to be fine-grained positioning, and the total number of computational units with an angle greater than or equal to 45° that are determined to be partial region segmentation, are counted, and the proportion of each side in the total number of units in the retained network is calculated.

[0091] The preset equilibrium ratio range for operator distribution within the network is defined as 40% to 60%. When a category biased towards either side, such as an extraction module focused on fine-grained localization, reaches 70% of the total, a compensation-based reverse correction logic is triggered. A fixed-proportion attenuation is applied to the current round's overall contribution value of all operators within the larger category, for example, by uniformly multiplying it by a penalty constant of 0.85, reducing its overall contribution in subsequent iterations. The removed computational units belonging to the smaller category, such as those focused on region segmentation, are sorted, and the top 5% of computational units are reconnected to the network and reassigned weights.

[0092] The aforementioned pruning and penalty balancing mechanism is continuously executed as the iteration progresses. Once the network structure stabilizes or the termination condition of the maximum period is reached, the parameters of the remaining computational units and network connection topology within the search system are saved and the code is reconstructed. This results in an optimal subnetwork with lower computational cost and balanced functionality.

[0093] S7 utilizes the optimal sub-network structure to construct a skew recognition model, inputs the image to be tested into the skew recognition model for processing, and outputs the lateral skew angle of the dump truck body.

[0094] A global average pooling layer and a linear fully connected layer are added to the end of the optimal sub-network structure to construct a complete end-to-end skew recognition model. The model parameters are updated using gradient descent on the dump truck operation training set, utilizing the Adam optimization algorithm and the mean squared error loss function.

[0095] The images of the test subject, acquired from the actual scene, are resized and preprocessed using tensor quantization before being input into a skew recognition model loaded with trained weights. Feature mappings are calculated through forward propagation within the model, and the lateral skew angle of the dump truck body is output via regression by a linear fully connected layer.

[0096] In the description of this specification, "multiple" or "several" means at least two, such as two, three or more, unless otherwise expressly and specifically defined.

Claims

1. A visual recognition method for detecting tilting during the lifting of a dump truck body, characterized in that, include: S1. Obtain a continuous video image frame sequence of the dump truck lifting process and construct a hypernet search space containing candidate computing units; S2. For any candidate computing unit in the hypernet search space, calculate the Shannon entropy of the difference matrix between adjacent frames based on the temporal feature map of the candidate computing unit to obtain the temporal difference entropy representing the motion complexity. S3. Based on the increase in loss function caused by removing candidate computation units in corner regression auxiliary tasks and pixel segmentation auxiliary tasks, construct a task preference vector representing functional bias; S4. Generate a prior mask based on the task preference vector, and extract the activation response matrix norm after convolving the prior mask with the image feature map to obtain a structural sensitivity scalar representing the spatial feature extraction capability; S5. Use a weighting function with the mean of global temporal difference entropy as the independent variable to combine the normalized temporal difference entropy, structural sensitivity scalar, and the magnitude of the task preference vector to obtain the comprehensive contribution of the candidate computation units; S6. Based on the comprehensive contribution and the balance constraint on the distribution of task preference vectors, select a fixed path in the hypernetwork search space to determine the optimal subnetwork structure; S7. Use the optimal subnetwork structure to construct a skew recognition model, input the image to be tested into the skew recognition model for processing, and output the lateral skew angle of the dump truck body.

2. The method according to claim 1, characterized in that, The process of acquiring a continuous video image frame sequence of the dump truck lifting process and constructing a hypernetic search space containing candidate computing units includes: A residual network with a preset depth is used as the backbone network architecture. Multiple convolutional kernels with different receptive field sizes and skip connections are set in parallel at each node of the backbone network architecture as candidate computational units. By constructing all possible directed connection combinations between nodes at each level, a hypernetwork search space covering multiple receptive field scales and feature fusion paths is obtained.

3. The method according to claim 1, characterized in that, The Shannon entropy of the difference matrix between adjacent frames is calculated based on the temporal feature map of the candidate computation unit to obtain the temporal difference entropy representing the motion complexity, including: The feature maps of two adjacent frames output when a continuous video image frame sequence passes through the candidate computation unit are obtained, and the absolute difference operation is performed on the two frame feature maps to obtain the difference matrix. Hard thresholding is used to denoise the difference matrix using a preset threshold, removing noise elements whose absolute value is less than the threshold and retaining non-zero elements that represent the effective motion region. Divide the numerical distribution of non-zero elements into a preset number of intervals, and calculate the probability ratio of the number of non-zero elements in each interval to the total number of non-zero elements. The information entropy is calculated based on each probability ratio, and the temporal differential entropy is used as a candidate computational unit.

4. The method according to claim 1, characterized in that, The step of constructing a task preference vector representing functional bias based on the increase in loss function caused by removing candidate computation units in corner regression auxiliary tasks and pixel segmentation auxiliary tasks includes: Calculate the relative increase of the first loss function of the corner regression auxiliary task and the relative increase of the second loss function of the pixel segmentation auxiliary task when removing candidate computation units compared to when including candidate computation units. The relative increase is the ratio of the absolute loss increase to the baseline loss value of the corresponding auxiliary task, and set the relative increase that is less than zero to zero. A task preference vector is constructed based on the relative increase of the first loss function and the relative increase of the second loss function.

5. The method according to claim 4, characterized in that, The step of constructing a task preference vector based on the relative increase of the first loss function and the relative increase of the second loss function includes: Using the relative increase of the first loss function as the first coordinate value and the relative increase of the second loss function as the second coordinate value, a two-dimensional vector is constructed in the preset two-dimensional task coordinate system. The two-dimensional vector is used as the task preference vector. The magnitude represents the overall contribution of the corner regression auxiliary task and the pixel segmentation auxiliary task. The angle between the vector and the coordinate axis of the two-dimensional task coordinate system represents the functional bias towards fine localization or region segmentation.

6. The method according to claim 1, characterized in that, The generation of the prior mask based on the task preference vector includes: Extract the angle between the task preference vector and the corresponding coordinate axis of the corner regression auxiliary task; When the included angle is less than a preset angle threshold, a two-dimensional high-pass filter with a preset spatial size that focuses on enhancing the high-frequency contour edge features is generated as a priori mask. When the included angle is greater than or equal to a preset angle threshold, a two-dimensional low-pass filter with a preset spatial size is generated as a priori mask, which focuses on smoothly extracting features of low-frequency flat regions.

7. The method according to claim 1, characterized in that, The step of extracting the activation response matrix norm after convolving the prior mask with the image feature map to obtain a structure sensitivity scalar representing the spatial feature extraction capability includes: Obtain the spatial feature map generated by the candidate computation unit on the single-frame search validation set image; The prior mask and the spatial feature map are subjected to a sliding convolution operation in the spatial dimension to generate an activation response matrix representing the degree of enhancement of the corresponding features. Calculate the L2 norm of the activation response matrix, and normalize the calculated L2 norm result as a structural sensitivity scalar.

8. The method according to claim 1, characterized in that, The method utilizes a weighting function with the mean of the global temporal difference entropy as the independent variable to combine the normalized temporal difference entropy, the structural sensitivity scalar, and the magnitude of the task preference vector to obtain the comprehensive contribution of the candidate computing unit, including: Calculate the average of the temporal differential entropy of all candidate computing units as the global temporal differential entropy mean; construct a weighted function containing a first weight coefficient and a second weight coefficient, wherein the first weight coefficient is positively correlated with the global temporal differential entropy mean, and the second weight coefficient is negatively correlated with the global temporal differential entropy mean and is subject to a non-negative lower bound constraint; By using a weighting function to combine the normalized temporal difference entropy, structural sensitivity scalar, and task preference vector magnitudes, the comprehensive contribution of candidate computing units is obtained.

9. The method according to claim 8, characterized in that, The method of combining the normalized temporal difference entropy, structural sensitivity scalar, and task preference vector magnitude using a weighting function to obtain the comprehensive contribution of candidate computing units includes: The time-series difference entropy after normalization is weighted using the first weighting coefficient, the structural sensitivity scalar is weighted using the second weighting coefficient, and then nonlinearly summed with the magnitude of the normalized task preference vector to obtain the comprehensive contribution.

10. The method according to claim 1, characterized in that, The step of selecting a fixed path in the hypernetwork search space to determine the optimal subnetwork structure based on the comprehensive contribution and the balance constraint on the task preference vector distribution includes: In each round of the search iteration, candidate computational units and related connection paths with a comprehensive contribution below a preset threshold are removed; The angle distribution of the task preference vectors of the retained candidate computing units is statistically analyzed. If the proportion of units that focus on fine positioning or region segmentation exceeds the preset equilibrium proportion range, the comprehensive contribution of the majority units is penalized, and the connection paths of the removed minority units are restored. When the network structure converges or reaches the maximum number of iterations, the retained candidate computational units and connection paths are solidified to obtain the optimal subnetwork structure.

Images