A high-speed road sweeper target recognition method and system based on machine vision

Abstract

The present invention provides a machine vision-based target recognition method and system for high-speed road sweepers. The method comprises the following steps: S1: acquiring real-time image data of the vehicle's operating area, wherein the acquired real-time image data carries the vehicle's real-time speed information; S2: performing adaptive block processing on the acquired real-time image data, dividing the real-time image data into multiple sub-image blocks; performing image enhancement processing on the acquired sub-image blocks, and obtaining enhanced image data based on the enhanced sub-image blocks; S3: processing the enhanced image data using a trained target recognition model based on the enhanced image data to obtain target recognition results in the operating area. The present invention helps improve the clarity of images of the road sweeper's operating area, thereby enhancing target recognition effectiveness.

Description

Technical Field

[0001] The present invention relates to the technical field of high-speed sweeping technology using machine vision, and in particular to a target recognition method and system for a high-speed sweeper based on machine vision. Background Art

[0002] At present, machine vision technology has been widely used in intelligent sweepers, enabling them to carry out targeted cleaning of garbage ahead of the operation route, thereby improving the intelligence level of automatic cleaning of the sweepers.

[0003] However, in current intelligent cleaning processes, when the sweeper's speed exceeds a certain level, the captured images can become blurred, reducing the accuracy of subsequent garbage identification. However, if a targeted, high-performance recognition model were implemented, the amount of data processed or transmitted during the intelligent sweeper's mission would be significantly increased, impacting the sweeper's sustained performance. Therefore, a machine vision target recognition method suitable for high-speed sweepers is highly desirable. Summary of the Invention

[0004] In response to the above problems, the present invention aims to provide a high-speed road sweeper target recognition method and system based on machine vision.

[0005] The purpose of the present invention is achieved by adopting the following technical solutions:

[0006] In a first aspect, the present invention proposes a high-speed road sweeper target recognition method based on machine vision, comprising the following steps:

[0007] S1 acquires real-time image data of the vehicle operation area, wherein the acquired real-time image data carries real-time speed information of the vehicle;

[0008] S2 performs adaptive block processing on the acquired real-time image data to divide the real-time image data into a plurality of sub-image blocks; performs image enhancement processing on the acquired sub-image blocks, and obtains enhanced image data based on the enhanced sub-image blocks;

[0009] Based on the enhanced image data, S3 uses the trained target recognition model to process the enhanced image data and obtain the target recognition results of the working area.

[0010] Preferably, step S1 includes:

[0011] Obtain the real-time speed information v(t) of the sweeper based on the onboard IMU;

[0012] Collect real-time image data X(t) of the sweeper operation area based on the camera;

[0013] Integrate real-time velocity information into real-time image data.

[0014] Preferably, step S2 specifically includes:

[0015] S21 extracts a real-time image frame and corresponding real-time speed information from the acquired real-time image data;

[0016] S22 performs adaptive block processing based on the acquired real-time image frame to divide the real-time image frame into multiple sub-image blocks;

[0017] S23 performs image enhancement processing on each sub-image block to obtain an enhanced sub-image block;

[0018] S24 reconstructs the enhanced sub-image block to obtain an enhanced real-time image frame;

[0019] Preferably, in step S21, real-time image frames are extracted from the real-time image data according to a set time interval.

[0020] Preferably, in step S22, adaptive block processing is performed based on the acquired real-time image frame, specifically including:

[0021] Divide the real-time image frame into N sub-image blocks of equal shape and area;

[0022] 1) For each sub-image block, calculate the complexity factor of the current sub-image block;

[0023] 2) Calculate the partition parameter value of the sub-image block according to the complexity factor and current speed information of the current sub-image block;

[0024] 3) Compare the partition parameter value of the current sub-image block with the set partition threshold value. When the partition parameter value of the sub-image block is greater than the set value, the current sub-image block is further divided into sub-image blocks with equal shape and area. M sub-image blocks;

[0025] Repeat the above steps 1) to 3) until there are no more sub-image blocks that can be further divided or the size of the sub-image block is smaller than the preset minimum size, thereby completing the sub-image block division of the real-time image frame.

[0026] Preferably, in step S22, the complexity factor calculation function used is:

[0027] ;

[0028] Where, C i Indicates the i sub-image blocks φ i The complexity factor, N i Indicates the iThe total number of pixels in the sub-image blocks, (x, y) ∈ φ i Represents pixel points (x, y) For sub-image blocks φ i Pixels in ; Sobel(x, y) Indicates that the pixel is obtained by Sobel operator (x, y) is the Sobel eigenvalue extracted from the center; λ Represents the set balance factor, where λ ∈ [0.4, 0.6] ; LBP(x, y) Indicates that the LBP operator is used to calculate the pixel points (x, y) is the local binary eigenvalue extracted from the center;

[0029] The partition parameter value calculation function used is:

[0030] ;

[0031] Where, S i Indicates the i sub-image blocks φ i The partition parameter value, v(t) Indicates real-time speed, v max Indicates the set maximum speed, C i Indicates the i sub-image blocks φ i The complexity factor, C TH Indicates the set complexity factor threshold.

[0032] Preferably, in step S23, image enhancement processing is performed on each sub-image block, specifically including:

[0033] According to the gray value of each pixel in the sub-image block, the sub-image block is obtained. φ i Grayscale statistical histogram of T i = {p(n)} ,in p(n) Indicates the i The gray value of the sub-image block is n The statistical probability value of the pixel point;

[0034] For the obtained grayscale statistical histogram T i Perform clipping, where the clipping function used is:

[0035] ;

[0036] in, p cro (n) Indicates the statistical probability value of the pixel with gray value n after clipping, which indicates the clipping threshold. , , S i Indicates the i The total number of pixels in the sub-image blocks, γ represents the localization factor, where γ ∈ [0.01, 0.05] , α i represents the dynamic limiting factor, v(t) Indicates real-time speed, v max Indicates the set maximum speed, C i Indicates the i sub-image blocks φ i The complexity factor, C TH Indicates the set complexity factor threshold; d i Indicates the i The pixel distance from the center point of each sub-image block to the center point of the image; D Represents the diagonal pixel distance of the image; μ Represents the set limit adjustment parameters, where μ ∈ [0.5, 2] ;

[0037] Compensation processing is performed on the cropped grayscale statistical histogram, and the compensation processing function used is:

[0038] ;

[0039] Where, p cps (n) Indicates that the gray value after compensation is n The statistical probability value of the pixel point; p cro (n) Indicates that the grayscale value after clipping is n The statistical probability value of the pixel point; j Represents a variable, p(j) Indicates the gray value is j The statistical probability value of the pixel point, β represents the clipping threshold;

[0040] Grayscale equalization is performed on the sub-image block according to the compensated grayscale statistical histogram to obtain the enhanced sub-image block. The equalization function used is:

[0041] ;

[0042] Where, Indicates the i The updated grayscale value of the pixel with grayscale value v in the sub-image block after grayscale equalization processing is completed; round represents the rounding function, S i No. i The total number of pixels in the sub-image blocks, CDF(v) Indicates that the grayscale value in the grayscale statistical histogram after compensation is v The cumulative distribution statistic of .

[0043] Preferably, in step S24, after the enhancement processing of each sub-image block is completed respectively, the enhanced sub-image blocks are merged to obtain a completed enhanced real-time image frame.

[0044] Preferably, in step S3, the enhanced real-time image frame is input into a trained target recognition model to process the enhanced image data, and the target recognition model outputs a corresponding garbage recognition result;

[0045] Among them, the target recognition model is built based on the MobileNetv3 network, including an input layer, a first convolutional layer, a bottleneck layer, and an output layer; the input layer is used to input enhanced real-time image frames, and the first convolutional layer uses a normal convolution kernel, where the convolution kernel size is 3×3, the step size is 2, and the activation function is ReLU; the bottleneck layer contains multiple (for example, 15) stacked convolution modules, where the depth convolution kernel used is 3×3 or 5×5, the step size is 1 or 2, the activation function is ReLU, and the expansion ratio is 1-6; the convolution module is also equipped with an attention module; the output layer contains a fully connected module, where the fully connected module uses a global average pooling operation, the classification function uses the Softmax function, and outputs garbage recognition results.

[0046] In a second aspect, the present invention proposes a high-speed road sweeper target recognition system based on machine vision, comprising:

[0047] An acquisition module is used to acquire real-time image data of the vehicle operation area, wherein the acquired real-time image data carries the real-time speed information of the vehicle;

[0048] An enhancement module is used to perform adaptive block processing based on the acquired real-time image data, dividing the real-time image data into multiple sub-image blocks; performing image enhancement processing on the acquired sub-image blocks, and obtaining enhanced image data based on the enhanced sub-image blocks;

[0049] The recognition module is used to process the enhanced image data based on the enhanced image data using a trained target recognition model to obtain target recognition results in the operating area.

[0050] The present invention provides a machine vision-based target recognition method and system for high-speed road sweepers. This method integrates the real-time speed information of the road sweeper into the corresponding real-time image data collected from the work area. Adaptive block processing is performed based on the acquired real-time speed information and the image's characteristic regions. Targeted enhancement processing is then performed based on the sub-image blocks, thereby improving the clarity of detailed features in the image. Finally, target recognition within the work area is completed using an image processing model based on the enhanced image data.

[0051] In response to the situation where images captured by intelligent road sweepers at high moving speeds are prone to image blur, the above-mentioned embodiment of the present invention, through the proposed enhancement processing method, can first adaptively block the part of the image where features are concentrated, wherein speed information is further added in the block process to intelligently adjust the size of the block, thereby performing targeted enhancement processing on the feature area. At the same time, a sub-image block-based image enhancement processing method is proposed, which can adaptively enhance the sub-image block based on speed characteristics and complexity characteristics, thereby improving the performance of local detail features in the image. This improves the accuracy of subsequent target recognition based on image data, and effectively improves the data processing efficiency of the enhancement processing, which can adapt to target recognition tasks during high-speed road sweeper operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The present invention is further described with reference to the accompanying drawings. However, the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative effort.

[0053] Figure 1 This is a flowchart of a method for identifying a target of a high-speed road sweeper based on machine vision according to an embodiment of the present invention;

[0054] Figure 2 for Figure 1 Flow chart of the specific steps of step S2 in the embodiment;

[0055] Figure 3 This is a framework diagram of a high-speed road sweeper target recognition system based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention is further described in conjunction with the following application scenarios.

[0057] See also Figure 1 , which shows a high-speed sweeper target recognition method based on machine vision, including the following steps:

[0058] S1 acquires real-time image data of the vehicle operation area, wherein the acquired real-time image data carries real-time speed information of the vehicle;

[0059] S2 performs adaptive block processing on the acquired real-time image data to divide the real-time image data into a plurality of sub-image blocks; performs image enhancement processing on the acquired sub-image blocks, and obtains enhanced image data based on the enhanced sub-image blocks;

[0060] Based on the enhanced image data, S3 uses the trained target recognition model to process the enhanced image data and obtain the target recognition results of the working area.

[0061] The above-described embodiment of the present invention proposes a machine vision-based target recognition method for high-speed road sweepers. This method integrates the real-time speed information of the road sweeper into the corresponding real-time image data collected from the work area. Adaptive block processing is performed based on the acquired real-time speed information and the characteristic regions of the image. Targeted enhancement processing is then performed based on the sub-image blocks, thereby improving the clarity of detailed features in the image. Finally, the enhanced image data is used to complete target recognition in the work area through an image processing model.

[0062] In an exemplary scenario, the above-mentioned machine vision-based high-speed road sweeper target recognition method and system proposed in the present invention can be built based on the built-in data processing equipment or cloud server of the high-speed road sweeper.

[0063] Preferably, step S1 includes:

[0064] Obtain the real-time speed information v(t) of the sweeper based on the onboard IMU;

[0065] Collect real-time image data X(t) of the sweeper operation area based on the camera;

[0066] Integrate real-time velocity information into real-time image data.

[0067] In the above embodiment of the present invention, a camera is positioned in front of the sweeper to focus on the sweeper's operating area, and the camera collects real-time image data of the sweeper's operating area. Simultaneously, the sweeper's built-in IMU can simultaneously collect real-time speed data.

[0068] Preferably, see Figure 2 , step S2 includes:

[0069] S21 extracts a real-time image frame X(t) and corresponding real-time speed information v(t) from the acquired real-time image data;

[0070] S22 performs adaptive block processing based on the acquired real-time image frame to divide the real-time image frame into multiple sub-image blocks;

[0071] S23 performs image enhancement processing on each sub-image block to obtain an enhanced sub-image block;

[0072] S24 reconstructs the enhanced real-time image frame X according to the enhanced sub-image block ’ (t).

[0073] The above-mentioned embodiment of the present invention addresses the problem of image blurring that easily occurs when images collected by an intelligent road sweeper are moving at a high speed. The above-mentioned embodiment of the present invention, through the proposed enhancement processing method, can first adaptively block the portion of the image where features are concentrated, wherein speed information is further added in the block process to intelligently adjust the size of the blocks, thereby performing targeted enhancement processing on the feature areas. At the same time, a sub-image block-based image enhancement processing method is proposed, which can adaptively enhance the sub-image blocks based on speed features and complexity features, thereby improving the performance of local detail features in the image. This improves the accuracy of subsequent target recognition based on image data, and effectively improves the data processing efficiency of the enhancement processing, which can adapt to target recognition tasks during the operation of high-speed road sweepers.

[0074] Preferably, in step S21, real-time image frames are extracted from the real-time image data according to a set time interval.

[0075] In an exemplary scenario, the time interval can be set to 16-200ms based on the comprehensive computing power and the response speed of the street sweeper.

[0076] Preferably, in step S22, adaptive block processing is performed based on the acquired real-time image frame, specifically including:

[0077] Initialize the number of division layers T=1 and divide the real-time image frame into N sub-image blocks of equal shape and area;

[0078] 1) For each sub-image block, calculate the complexity factor of the current sub-image block, where the complexity factor calculation function used is:

[0079]

[0080] Where, C i Indicates the i sub-image blocks φ i The complexity factor,N i Indicates the i The total number of pixels in the sub-image blocks, (x, y) ∈ φ i Represents pixel points (x, y) For sub-image blocks φ i Pixels in ; Sobel(x, y) Indicates that the pixel is obtained by Sobel operator (x, y) is the Sobel eigenvalue extracted from the center; λ Represents the set balance factor, where λ ∈ [0.4, 0.6] ; LBP(x, y) Indicates that the LBP operator is used to calculate the pixel points (x, y) is the local binary eigenvalue extracted from the center;

[0081] 2) Calculate the partition parameter value of the sub-image block according to the complexity factor and current speed information of the current sub-image block, wherein the partition parameter value calculation function used is:

[0082]

[0083] Where, S i Indicates the i sub-image blocks φ i The partition parameter value, v(t) Indicates real-time speed, v max Indicates the set maximum speed, C i Indicates the i sub-image blocks φ i The complexity factor, C TH Indicates the set complexity factor threshold;

[0084] 3) According to the partition parameter value of the current sub-image block and the set partition threshold S TH When the partition parameter value of the sub-image block is compared S i >S TH When , the current sub-image block is further divided into blocks with equal shape and area. M sub-image blocks;

[0085] Update the decomposition level T=T+1 and repeat the above steps 1)-3) until there are no more sub-image blocks that can be further divided or the size of the sub-image block is smaller than the preset minimum size, thus completing the sub-image block division of the real-time image frame.

[0086] Among them, when the image is divided for the first time, you can set N The value is 1-9, and can be set according to actual needs during subsequent division. M The value is 1-9.

[0087] The specific calculation method of Sobel eigenvalue is to align the pixel points with Sobel operators in the vertical and horizontal directions respectively. (x, y) Then calculate the vertical Sobel eigenvalues G x (x, y) and horizontal Sobel eigenvalues G y (x, y) , further comprehensive calculation to obtain the Sobel eigenvalue ;

[0088] Among them, the calculation method of local binary eigenvalue is ,in s(h(x, y) - h(i)) represents the threshold function, when (h(x, y) - h(i)) > 0 , s(h(x, y) - h(i)) = 1 ,otherwise s(h(x, y) - h (i))=0 , h(x, y) Represents pixel points (x, y) The grayscale eigenvalue of h(i) Indicated by pixels (x, y) The center i The grayscale eigenvalues of the neighboring pixels.

[0089] Preferably, the complexity factor threshold can be set based on experience C TH The range is 20-70 , the best choice C TH = 50 ;

[0090] Maximum speed v max The range can be set to 15 - 20 m / s ,Right now 18 - 72 km / h .

[0091] In the above-described embodiment of the present invention, when adaptively segmenting the extracted image frame, the feature complexity of the image is calculated based on the edge features and texture features of each sub-image block, and the feature complexity is used to represent the dispersion and aggregation level of feature information in the image. The sub-image blocks are further divided based on the complexity factor and real-time speed characteristics. When the speed is faster or the complexity is higher, the image blocks are divided into finer sizes for targeted enhancement processing (which helps to restore the detail features destroyed by motion blur and improve the representation level of feature details during the subsequent enhancement process), thereby improving the refinement of the enhancement processing. At the same time, for general areas, subsequent enhancement processing is based on larger image blocks, thereby avoiding enhancement redundancy and ensuring the efficiency of the enhancement processing. Through this adaptive sub-image block segmentation method, the effect of subsequent enhancement processing and the amount of data processing can be balanced.

[0092] Preferably, in step S23, image enhancement processing is performed on each sub-image block, specifically including:

[0093] According to the gray value of each pixel in the sub-image block, the sub-image block is obtained. φ i Grayscale statistical histogram of T i = {p(n)} ,in p(n) Indicates the i The gray value of the sub-image block is n The statistical probability value of the pixel point;

[0094] For the obtained grayscale statistical histogram T i Perform clipping, where the clipping function used is:

[0095]

[0096] in, p cro (n) Indicates the statistical probability value of the pixel with gray value n after clipping, which indicates the clipping threshold. , , S i Indicates the i The total number of pixels in the sub-image blocks, γ represents the localization factor, where γ ∈ [0.01, 0.05] , α i represents the dynamic limiting factor, v(t) Indicates real-time speed, v max Indicates the set maximum speed,C i Indicates the i sub-image blocks φ i The complexity factor, C TH Indicates the set complexity factor threshold; d i Indicates the i The pixel distance from the center point of each sub-image block to the center point of the image; D Represents the diagonal pixel distance of the image; μ Represents the set limit adjustment parameters, where μ ∈ [0.5, 2] ;

[0097] Compensation processing is performed on the cropped grayscale statistical histogram, and the compensation processing function used is:

[0098]

[0099] Where, p cps (n) Indicates that the gray value after compensation is n The statistical probability value of the pixel point; p cro (n) Indicates that the grayscale value after clipping is n The statistical probability value of the pixel point; j Represents a variable, p(j) Indicates the gray value is j The statistical probability value of the pixel point, β represents the clipping threshold;

[0100] Grayscale equalization is performed on the sub-image block according to the compensated grayscale statistical histogram to obtain the enhanced sub-image block. The equalization function used is:

[0101]

[0102] Where, Indicates the i The updated grayscale value of the pixel with grayscale value v in the sub-image block after grayscale equalization processing is completed; round represents the rounding function, S i No. i The total number of pixels in the sub-image blocks, CDF(v) Indicates that the grayscale value in the grayscale statistical histogram after compensation is v The cumulative distribution statistic of .

[0103] Among them, the above d iThe setting can also represent the distance from the sub-image block to the center point of the working area according to actual conditions, wherein the working area or the center point of the working area can be pre-calibrated according to the setting position or picture of the camera to complete the setting.

[0104] The above-mentioned embodiment of the present invention, after completing the division of sub-image blocks, also proposes a targeted image enhancement processing for sub-image blocks, wherein first, a grayscale statistical histogram is statistically calculated for the grayscale feature information in the image, and the grayscale converged part of the histogram is targetedly clipped and compensated. In the process of histogram clipping, the influence of speed characteristics on the grayscale characteristics of the image is specially considered (the image blur caused by high-speed movement makes the grayscale features of the image more converged, resulting in reduced contrast in the detail parts), the clipping effect is intelligently adjusted, and the image is further grayscale equalized, and the grayscale of the blurred part and the feature detail part is targetedly stretched, thereby effectively repairing the blurred area in the image and highlighting the feature detail parts.

[0105] Preferably, in step S24, after the enhancement processing of each sub-image block is completed respectively, the enhanced sub-image blocks are merged to obtain a completed enhanced real-time image frame.

[0106] After completing the enhancement processing of each sub-image, the sub-image blocks are merged to obtain an enhanced real-time image frame as the basis for subsequent target recognition.

[0107] Preferably, in step S3, the enhanced real-time image frame is input into a trained target recognition model to process the enhanced image data, and the target recognition model outputs a corresponding garbage recognition result.

[0108] Preferably, the target recognition model is built based on the MobileNetv3 network, including an input layer, a first convolutional layer, a bottleneck layer, and an output layer; the input layer is used to input enhanced real-time image frames, and the first convolutional layer adopts an ordinary convolution kernel, wherein the convolution kernel size is 3×3, the step size is 2, and the activation function is ReLU; the bottleneck layer contains multiple (for example, 15) stacked convolution modules, wherein the depth convolution kernel used is 3×3 or 5×5, the step size is 1 or 2, the activation function is ReLU, and the expansion ratio is 1-6; wherein the convolution module is also provided with an attention module; the output layer contains a fully connected module, wherein the fully connected module adopts a global average pooling operation, the classification function adopts a Softmax function, and outputs garbage recognition results.

[0109] In an optional embodiment, the specific structure of the MobileNetv3 network is as follows: INPUT: input enhanced real-time image frame; Conv2D: convolution kernel size is 3×3, step size is 2, activation function is ReLU; Bottleneck1: expansion ratio is 1, convolution kernel is 3×3, step size is 1, activation function is ReLU, and an attention module is provided; Bottleneck2: expansion ratio is 4, convolution kernel is 3×3, step size is 2, activation function is ReLU; Bottleneck3: expansion ratio is 3, convolution kernel is 3×3, step size is 1, activation function is ReLU; Bottleneck4: expansion ratio is 3, convolution kernel is 5×5, stride is 2, activation function is ReLU, and an attention module is set; Bottleneck5: expansion ratio is 3, convolution kernel is 5×5, stride is 1, activation function is ReLU, and an attention module is set; Bottleneck6: expansion ratio is 3, convolution kernel is 5×5, stride is 1, activation function is ReLU, and an attention module is set; Bottleneck7: expansion ratio is 6, convolution kernel is 3×3, stride is 2, activation function is ReLU; Bottleneck8: expansion ratio is 2, convolution kernel is 3×3, stride is 1, activation function is ReLU; Bottleneck9: expansion ratio is 2, convolution kernel is 3×3, stride is 1, activation function is ReLU; Bottleneck10: expansion ratio is 2, convolution kernel is 3×3, stride is 1, activation function is ReLU; Bottleneck11: expansion ratio is 2, convolution kernel is 3×3, stride is 1, activation function is ReLU The expansion ratio is 6, the convolution kernel is 3×3, the stride is 1, the activation function is ReLU, and an attention module is set; Bottleneck12: the expansion ratio is 6, the convolution kernel is 3×3, the stride is 1, the activation function is ReLU, and an attention module is set; Bottleneck13: the expansion ratio is 6, the convolution kernel is 5×5, the stride is 2, the activation function is ReLU, and an attention module is set; Bottleneck14: the expansion ratio is 6, the convolution kernel is 5×5, the stride is 1, the activation function is ReLU, and an attention module is set; Bottleneck15: the expansion ratio is 6, the convolution kernel is 5×5, the stride is 1, the activation function is ReLU, and an attention module is set; Conv2D: the convolution kernel is 1×1; AvgPool: average pooling operation, the pooling kernel is 7×7; Conv2D: the convolution kernel is 1×1; Conv2D: classifier, the classification function is Softmax; OUTPUT: output classification results.

[0110] Preferably, for the training of the target recognition model, the model can be trained by constructing a standard data set (including image data and corresponding recognition results), and after the training is completed, the accuracy test can be completed through the test set to obtain a trained target recognition model.

[0111] The above-mentioned embodiment of the present invention also proposes a target recognition model based on the MobileNetv3 network. By using the lightweight MobileNetv3 network as a basis, it can realize the accurate recognition of targets (such as garbage, etc.) in images and improve the target recognition speed and accuracy.

[0112] Preferably, the method further comprises:

[0113] According to the target recognition results, the cleaning system is controlled to complete the targeted cleaning tasks.

[0114] After completing the target identification in the working area, when garbage or other targets that need to be cleaned are identified, the cleaning equipment mounted on the sweeper is further controlled to complete the cleaning of the target to complete the target cleaning task.

[0115] See also Figure 3 The embodiment shows a high-speed road sweeper target recognition system based on machine vision, including:

[0116] An acquisition module is used to acquire real-time image data of the vehicle operation area, wherein the acquired real-time image data carries the real-time speed information of the vehicle;

[0117] An enhancement module is used to perform adaptive block processing based on the acquired real-time image data, dividing the real-time image data into multiple sub-image blocks; performing image enhancement processing on the acquired sub-image blocks, and obtaining enhanced image data based on the enhanced sub-image blocks;

[0118] The recognition module is used to process the enhanced image data based on the enhanced image data using a trained target recognition model to obtain target recognition results in the operating area.

[0119] Preferably, the system further includes a control module for controlling the cleaning system to complete targeted cleaning tasks according to the target recognition results.

[0120] At the same time, each module in the above-mentioned high-speed sweeper target recognition system based on machine vision is also used to realize the above-mentioned Figure 1 The corresponding method steps in the machine vision-based high-speed road sweeper target recognition method are not repeated here in the present invention.

[0121] It should be noted that the functional units / modules in the various embodiments of the present invention may be integrated into a single processing unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated into a single unit / module. The aforementioned integrated units / modules may be implemented in the form of hardware or software functional units / modules.

[0122] From the above description of the embodiments, those skilled in the art will clearly understand that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code, or any appropriate combination thereof. For hardware implementation, the processor may be implemented in one or more of the following: an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a processor, a controller, a microcontroller, a microprocessor, or other electronic units designed to implement the functionality described herein, or any combination thereof. For software implementation, some or all of the processes of the embodiments may be performed by a computer program instructing the relevant hardware. During implementation, the program may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media includes any medium that facilitates the transfer of a computer program from one location to another. Storage media can be any available medium that can be accessed by a computer. Computer-readable media may include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should analyze that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A high-speed road sweeper target recognition method based on machine vision, characterized in that: The steps include: S1 acquires real-time image data of the vehicle operation area, wherein the acquired real-time image data carries real-time speed information of the vehicle; S2 performs adaptive block processing based on the acquired real-time image data, dividing the real-time image data into multiple sub-image blocks; Performing image enhancement processing on the acquired sub-image block and obtaining enhanced image data based on the enhanced sub-image block, specifically including: S21 extracts real-time image frames and corresponding real-time speed information from the acquired real-time image data; S22 performs adaptive block processing based on the acquired real-time image frame to divide the real-time image frame into multiple sub-image blocks; S23 performs image enhancement processing on each sub-image block to obtain an enhanced sub-image block; S24 reconstructs the enhanced sub-image block to obtain an enhanced real-time image frame; S3 processes the enhanced image data using the trained target recognition model to obtain the target recognition results of the operating area; Wherein, in step S22, adaptive block processing is performed based on the acquired real-time image frame, specifically including: Divide the real-time image frame into N sub-image blocks of equal shape and area; 1) For each sub-image block, calculate the complexity factor of the current sub-image block, where the complexity factor calculation function used is: ; Where, C i Indicates the i sub-image blocks φ i The complexity factor, N i Indicates the i The total number of pixels in the sub-image blocks, (x,y)∈φ i Represents pixel points (x,y) For sub-image blocks φ i Pixels in ; Sobel(x,y) Indicates that the pixel is obtained by Sobel operator (x,y) is the Sobel eigenvalue extracted from the center; λ Represents the set balance factor, where λ∈[0.4,0.6] ; LBP(x,y) Indicates that the LBP operator is used to calculate the pixel points (x,y) is the local binary eigenvalue extracted from the center; 2) Calculate the partition parameter value of the sub-image block according to the complexity factor and current speed information of the current sub-image block, wherein the partition parameter value calculation function used is: Where, S i Indicates the i sub-image blocks φ i The partition parameter value, v(t) Indicates real-time speed, v max Indicates the set maximum speed, C i Indicates the i sub-image blocks φ i The complexity factor, C TH Indicates the set complexity factor threshold; 3) According to the partition parameter value of the current sub-image block and the set partition threshold S TH When the partition parameter value of the sub-image block is compared S i >S TH When , the current sub-image block is further divided into blocks with equal shape and area. M sub-image blocks; Repeat the above steps 1) to 3) until there are no more sub-image blocks that can be further divided or the size of the sub-image block is smaller than the preset minimum size, thereby completing the sub-image block division of the real-time image frame.

2. The method for identifying a target of a high-speed road sweeper based on machine vision according to claim 1, characterized in that: Step S1 includes: Obtain real-time speed information of the sweeper based on the onboard IMU; Collect real-time image data of the sweeper's operating area based on the camera; Integrate real-time velocity information into real-time image data.

3. The method for identifying a target of a high-speed road sweeper based on machine vision according to claim 2, characterized in that: In step S21 , real-time image frames are extracted from the real-time image data according to a set time interval.

4. The method for identifying a target of a high-speed road sweeper based on machine vision according to claim 1, wherein: In step S23, image enhancement processing is performed on each sub-image block, specifically including: According to the gray value of each pixel in the sub-image block, the sub-image block is obtained. φ i Grayscale statistical histogram of T i = {p(n)} ,in p(n) Indicates the i The gray value of the sub-image block is n The statistical probability value of the pixel point; For the obtained grayscale statistical histogram T i Perform clipping, where the clipping function used is: ; in, p cro (n) Represents the statistical probability value of the pixel with gray value n after clipping. represents the clipping threshold, , , S i Indicates the i The total number of pixels in the sub-image blocks, γ represents the localization factor, where γ∈[0.01,0.05] , α i represents the dynamic limiting factor, v(t) Indicates real-time speed, v max Indicates the set maximum speed, C i Indicates the i sub-image blocks φ i The complexity factor, C TH Indicates the set complexity factor threshold; d i Indicates the i The pixel distance from the center point of each sub-image block to the center point of the image; D Represents the diagonal pixel distance of the image; μ Represents the set limit adjustment parameters, where μ∈[0.5,2] ; Compensation processing is performed on the cropped grayscale statistical histogram, and the compensation processing function used is: ; Where, p cps (n) Indicates that the gray value after compensation is n The statistical probability value of the pixel point; p cro (n) Indicates that the gray value after clipping is n The statistical probability value of the pixel point; j Represents a variable, p(j) Indicates the gray value is j The statistical probability value of the pixel point, β represents the clipping threshold; Grayscale equalization is performed on the sub-image block according to the compensated grayscale statistical histogram to obtain the enhanced sub-image block. The equalization function used is: ; Where, Indicates the i The updated grayscale value of the pixel with grayscale value v in the sub-image block after grayscale equalization processing is completed; round represents the rounding function, S i No. i The total number of pixels in the sub-image blocks, CDF(v) Indicates that the grayscale value in the grayscale statistical histogram after compensation is v The cumulative distribution statistic of .

5. The method for identifying a target of a high-speed road sweeper based on machine vision according to claim 4, characterized in that: In step S24, after the enhancement processing of each sub-image block is completed, the enhanced sub-image blocks are merged to obtain a completed enhanced real-time image frame.

6. The method for high-speed road sweeper target recognition based on machine vision according to claim 1, characterized in that: In step S3, the enhanced real-time image frame is input into the trained target recognition model to process the enhanced image data, and the target recognition model outputs the corresponding garbage recognition result; Among them, the target recognition model is built based on the MobileNetv3 network, including the input layer, the first convolutional layer, the bottleneck layer, and the output layer; the input layer is used to input enhanced real-time image frames, and the first convolutional layer uses a normal convolution kernel, where the convolution kernel size is 3×3, the step size is 2, and the activation function is ReLU; the bottleneck layer contains multiple stacked convolution modules, where the depth convolution kernel used is 3×3 or 5×5, the step size is 1 or 2, the activation function is ReLU, and the expansion ratio is 1-6; the convolution module is also equipped with an attention module; the output layer contains a fully connected module, where the fully connected module uses a global average pooling operation, the classification function uses the Softmax function, and outputs garbage recognition results.

7. A high-speed road sweeper target recognition system based on machine vision, characterized in that: include: An acquisition module is used to acquire real-time image data of the vehicle operation area, wherein the acquired real-time image data carries the real-time speed information of the vehicle; An enhancement module is used to perform adaptive block processing based on the acquired real-time image data, dividing the real-time image data into multiple sub-image blocks; performing image enhancement processing on the acquired sub-image blocks, and obtaining enhanced image data based on the enhanced sub-image blocks; The recognition module is used to process the enhanced image data using a trained target recognition model to obtain target recognition results in the operating area; The enhancement module specifically includes: Extracting real-time image frames and corresponding real-time speed information from the acquired real-time image data; Adaptively partitioning the image frame into multiple sub-image blocks based on the acquired real-time image frame; Performing image enhancement processing on each sub-image block to obtain an enhanced sub-image block; Reconstructing the enhanced sub-image blocks to obtain enhanced real-time image frames; The adaptive block processing based on the acquired real-time image frame specifically includes: Divide the real-time image frame into N sub-image blocks of equal shape and area; 1) For each sub-image block, calculate the complexity factor of the current sub-image block, where the complexity factor calculation function used is: ; Where, C i Indicates the i sub-image blocks φ i The complexity factor, N i Indicates the i The total number of pixels in the sub-image blocks, (x,y)∈φ i Represents pixel points (x,y) For sub-image blocks φ i Pixels in ; Sobel(x,y) Indicates that the pixel is obtained by Sobel operator (x,y) is the Sobel eigenvalue extracted from the center; λ Represents the set balance factor, where λ∈[0.4,0.6] ; LBP(x,y) Indicates that the LBP operator is used to calculate the pixel points (x,y) is the local binary eigenvalue extracted from the center; 2) Calculate the partition parameter value of the sub-image block according to the complexity factor and current speed information of the current sub-image block, wherein the partition parameter value calculation function used is: Where, S i Indicates the i sub-image blocks φ i The partition parameter value, v(t) Indicates real-time speed, v max Indicates the set maximum speed, C i Indicates the i sub-image blocks φ i The complexity factor, C TH Indicates the set complexity factor threshold; 3) According to the partition parameter value of the current sub-image block and the set partition threshold S TH When the partition parameter value of the sub-image block is compared S i >S TH When , the current sub-image block is further divided into blocks with equal shape and area. M sub-image blocks; Repeat the above steps 1) to 3) until there are no more sub-image blocks that can be further divided or the size of the sub-image block is smaller than the preset minimum size, thereby completing the sub-image block division of the real-time image frame.

Images