Road freight vehicle and cargo external dimensions detection and warning system based on AI system

Abstract

The present invention provides an AI-based system for detecting and warning the outer dimensions of highway freight vehicles and cargo. The system includes a vehicle detection and warning center, a vehicle image acquisition device, an image sampling unit, an image conversion unit, an image reconstruction unit, a dimension detection unit, and an early warning unit. The vehicle detection and warning center is connected to the vehicle image acquisition device, the image sampling unit, the image conversion unit, the image reconstruction unit, the dimension detection unit, and the early warning unit, respectively, to store and manage data from each unit or device. The present invention uses the image sampling unit, the image conversion unit, and the image reconstruction unit to perform resolution processing on an initial vehicle image to obtain a high-resolution viewpoint image. The viewpoint image is then subjected to AI intelligent detection using a vehicle and cargo outer dimension detection model in the dimension detection unit to obtain outer dimension detection information. Finally, an early warning can be issued based on the outer dimension detection information, thereby improving the accuracy of vehicle and cargo outer dimension detection and early warning.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an AI-based system for detecting and warning the outer dimensions of highway freight vehicles and cargo. Background Art

[0002] Detecting the overall dimensions of freight vehicles and cargo is a crucial step in ensuring traffic safety and road management. With the development of the logistics industry, the size and load capacity of freight vehicles have become increasingly prominent, potentially impacting road safety, bridge capacity, and traffic flow. Existing methods for detecting the overall dimensions of freight vehicles primarily rely on capturing images of the freight vehicles and then performing direct comparisons. These methods fail to consider the impact of different environments on the resolution of vehicle images, resulting in errors in the comparison results and low accuracy in warnings for these measurements. Summary of the Invention

[0003] The present invention provides an AI-based system for detecting and warning the outer dimensions of highway freight vehicles and cargo. The system combines artificial intelligence with high-resolution image processing to improve the accuracy of detection and warning of the outer dimensions of vehicles and cargo.

[0004] In a first aspect, the present invention provides an AI-based system for detecting and warning the outer dimensions of road freight vehicles and cargo, comprising a vehicle detection and warning center, a vehicle image acquisition device, an image sampling unit, an image conversion unit, an image reconstruction unit, a dimension detection unit, and an early warning unit; the vehicle detection and early warning center is connected to the vehicle image acquisition device, the image sampling unit, the image conversion unit, the image reconstruction unit, the dimension detection unit, and the early warning unit, respectively, to store and manage data from each unit or device;

[0005] A vehicle image acquisition device is used to acquire an initial vehicle image of a target freight vehicle passing through a preset highway section;

[0006] an image sampling unit, configured to downsample the initial vehicle image to obtain a first target vehicle image;

[0007] an image conversion unit, configured to convert the first target vehicle image into a second target vehicle image;

[0008] an image reconstruction unit, configured to input the central viewpoint image of the initial vehicle image and the second target vehicle image into an image reconstruction model to obtain a viewpoint image output by the image reconstruction model;

[0009] a size detection unit, configured to recognize the viewpoint image based on a preset vehicle and cargo outer dimension detection model, and obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model; the vehicle and cargo outer dimension detection model is trained based on sample viewpoint images and their corresponding outer dimension labels;

[0010] An early warning unit is used to issue an early warning based on the outer dimension detection information.

[0011] In a second aspect, the present invention further provides an AI-based method for detecting and warning the outer dimensions of a highway freight vehicle and cargo, which is applied to the AI-based method for detecting and warning the outer dimensions of a highway freight vehicle and cargo described in the first aspect. The AI-based method for detecting and warning the outer dimensions of a highway freight vehicle and cargo described in the first aspect comprises:

[0012] Collecting an initial vehicle image of a target freight vehicle passing through a preset highway section;

[0013] Downsampling the initial vehicle image to obtain a first target vehicle image;

[0014] converting the first target vehicle image into a second target vehicle image;

[0015] Inputting the central viewpoint image of the initial vehicle image and the second target vehicle image into an image reconstruction model to obtain a viewpoint image output by the image reconstruction model;

[0016] Recognizing the viewpoint image based on a preset vehicle and cargo outer dimension detection model to obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model; the vehicle and cargo outer dimension detection model is trained based on sample viewpoint images and their corresponding outer dimension labels;

[0017] An early warning is issued based on the outer dimension detection information.

[0018] According to the AI ​​system-based highway freight vehicle and cargo outer dimensions detection and early warning method provided by an embodiment of the present invention, the image reconstruction model includes a texture extractor layer, a disparity estimation network layer, a fusion upsampling layer, and a reverse mapping layer;

[0019] The disparity estimation network layer is used to input the second target vehicle image into the disparity estimation network to obtain an initial disparity map output by the disparity estimation network;

[0020] The texture extractor layer is used to input the central viewpoint images of the continuous initial vehicle images into the texture extractor, and obtain the feature map containing high-resolution image texture information output by the texture extractor;

[0021] The fusion upsampling layer is used to fuse and upsample the feature map and the initial disparity map to obtain a target disparity map;

[0022] The reverse mapping layer is used to obtain the viewpoint image based on the target disparity map.

[0023] According to the AI ​​system-based highway freight vehicle and cargo outer dimensions detection and early warning method provided by an embodiment of the present invention, the image reconstruction model training step includes:

[0024] Acquire an initial sample vehicle image, downsample the initial sample vehicle image to obtain a first target sample vehicle image, convert the first target sample vehicle image into a second target sample vehicle image, and determine an image model to be trained;

[0025] Inputting the central viewpoint image of the initial sample vehicle image and the second target sample vehicle image into the image model to be trained to obtain a predicted viewpoint image output by the image model to be trained;

[0026] Based on the difference between the initial sample vehicle image and the predicted viewpoint image, a target loss is determined, and based on the target loss, parameter iteration is performed on the image model to be trained to obtain the image reconstruction model.

[0027] According to the AI ​​system-based highway freight vehicle and cargo outer dimensions detection and early warning method provided by an embodiment of the present invention, the predicted viewpoint image includes a plurality of first predicted viewpoint images and a plurality of second predicted viewpoint images;

[0028] The plurality of first predicted viewpoint images are obtained one by one through reverse mapping based on the target predicted disparity map and the initial sample vehicle image; the plurality of second predicted viewpoint images are obtained one by one through reverse mapping based on the target predicted disparity map and the initial sample vehicle image; the first predicted viewpoint image is a forward predicted viewpoint image, and the second predicted viewpoint image is a reverse predicted viewpoint image; the forward direction is mapped from left to right, and the reverse direction is mapped from right to left; the target predicted disparity map is output by a fusion upsampling layer in the image model to be trained;

[0029] Accordingly, determining the target loss based on the difference between the initial sample vehicle image and the predicted viewpoint image includes:

[0030] determining a first loss based on a difference between the initial sample vehicle image and the first predicted viewpoint image;

[0031] determining a second loss based on a difference between the initial sample vehicle image and the second predicted viewpoint image;

[0032] The target loss is determined based on the first loss and the second loss.

[0033] According to an AI system-based early warning method for detecting the outer dimensions of a highway freight vehicle and cargo provided by an embodiment of the present invention, determining the target loss based on the first loss and the second loss includes:

[0034] Stacking the initial sample vehicle images in a height dimension to obtain a first stacked image;

[0035] stacking the predicted viewpoint images in a height dimension to obtain a second stacked image;

[0036] determining a stack loss based on a difference between the first stack image and the second stack image;

[0037] The target loss is determined based on the first loss, the second loss, and the stacking loss.

[0038] According to the AI ​​system-based highway freight vehicle and cargo outer dimensions detection and early warning method provided by an embodiment of the present invention, the texture extractor includes a multi-layer residual block and a first convolution layer, and the residual block sequentially includes a second convolution layer, a batch normalization layer, and an activation layer.

[0039] According to the AI ​​system-based highway freight vehicle and cargo outer dimensions detection and early warning method provided by an embodiment of the present invention, the disparity estimation network layer includes a horizontal space feature extraction module, a cost volume construction module, an aggregation module, and a regression module;

[0040] The horizontal spatial feature extraction module is used to extract spatial feature information in the horizontal direction of the second target vehicle image to obtain a first feature map;

[0041] The cost volume construction module is used to obtain a high-dimensional feature volume based on the disparity level, the hole rate, the filling value, and the first feature map;

[0042] The aggregation module is used to perform global aggregation on the high-dimensional feature volume to obtain a target aggregate;

[0043] The regression module is used to regress the target aggregate to obtain the initial disparity map.

[0044] In a third aspect, the present invention further provides an electronic device comprising: a memory for storing a computer software program; and a processor for reading and executing the computer software program, thereby implementing any of the above-described AI system-based methods for detecting and warning the outer dimensions of a road freight vehicle and cargo.

[0045] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, wherein the storage medium stores a computer software program, and when the computer software program is executed by a processor, it implements any of the above-mentioned methods for detecting and warning the outer dimensions of a road freight vehicle and cargo based on an AI system.

[0046] In a fifth aspect, the present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned AI system-based methods for detecting and warning the outer dimensions of a highway freight vehicle and its cargo.

[0047] The AI-based highway freight vehicle and cargo outer dimension detection and warning system provided in an embodiment of the present invention performs resolution processing on an initial vehicle image through an image sampling unit, an image conversion unit, and an image reconstruction unit to obtain a high-resolution viewpoint image. The viewpoint image is then subjected to AI intelligent detection by a vehicle and cargo outer dimension detection model in a dimension detection unit to obtain outer dimension detection information. Finally, an early warning can be issued based on the outer dimension detection information, thereby accurately issuing an early warning based on the artificial intelligence system combined with the high-resolution image, thereby improving the accuracy of vehicle and cargo outer dimension detection and early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 This is a schematic diagram of the structure of the AI-based highway freight vehicle and cargo outer dimensions detection and warning system provided by the present invention;

[0050] Figure 2 This is a flow chart of a method for detecting and warning the outer dimensions of a highway freight vehicle and cargo based on an AI system provided by the present invention;

[0051] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0054] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0055] Figure 1 Schematic diagram of the structure of the AI-based highway freight vehicle and cargo outer dimensions detection and warning system provided by the present invention. Figure 1 As shown, the AI-based highway freight vehicle and cargo external dimension detection and warning system includes a vehicle detection and warning center, a vehicle image acquisition device, an image sampling unit, an image conversion unit, an image reconstruction unit, a dimension detection unit, and a warning unit; the vehicle detection and warning center is connected to the vehicle image acquisition device, the image sampling unit, the image conversion unit, the image reconstruction unit, the dimension detection unit, and the warning unit, respectively, to store and manage the data of each unit or device.

[0056] Among them, the vehicle image acquisition equipment can be installed on a preset highway section, such as on an electronic camera bar on highway section A, on a speed detection bar, etc. Therefore, the vehicle image acquisition equipment can capture the initial vehicle image of the target freight vehicle passing through the preset highway section.

[0057] Optionally, the image sampling unit may downsample the initial vehicle image captured by the vehicle image capture device to obtain a first target vehicle image.

[0058] Optionally, the image conversion unit may convert the first target vehicle image into a second target vehicle image.

[0059] Optionally, the image reconstruction unit may input the central viewpoint image of the initial vehicle image and the second target vehicle image into the image reconstruction model to obtain the viewpoint image output by the image reconstruction model.

[0060] Optionally, the size detection unit may identify the viewpoint image according to a preset vehicle and cargo outer dimension detection model to obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model.

[0061] The vehicle and cargo outer dimensions detection model in the embodiment of the present invention is trained based on sample viewpoint images and their corresponding outer dimensions labels.

[0062] Optionally, the warning unit may issue a warning based on the external dimension detection information. Specifically, the warning unit obtains the external dimensions of the target freight vehicle and cargo based on the external dimension detection information, compares the external dimensions of the vehicle and cargo with preset dimensions, and issues a warning based on the comparison result. The preset dimensions are set based on actual conditions. If the external dimensions of the vehicle and cargo exceed the preset dimensions, a warning notification is sent to the user terminal.

[0063] In another embodiment, the license plate number of the target freight vehicle can also be recorded, and the license plate number and warning notification can be bound and sent to the user terminal. The process of obtaining the license plate number is not detailed here. For example, it can be recognized through optical character recognition (OCR), deep learning, template matching, etc.

[0064] The AI-based highway freight vehicle and cargo outer dimension detection and warning system provided in an embodiment of the present invention performs resolution processing on an initial vehicle image through an image sampling unit, an image conversion unit, and an image reconstruction unit to obtain a high-resolution viewpoint image. The viewpoint image is then subjected to AI intelligent detection by a vehicle and cargo outer dimension detection model in a dimension detection unit to obtain outer dimension detection information. Finally, an early warning can be issued based on the outer dimension detection information, thereby accurately issuing an early warning based on the artificial intelligence system combined with the high-resolution image, thereby improving the accuracy of vehicle and cargo outer dimension detection and early warning.

[0065] Optional, see Figure 2 , Figure 2 1 is a flow chart of a method for detecting and warning the outer dimensions of a highway freight vehicle and cargo based on an AI system provided by the present invention. The method for detecting and warning the outer dimensions of a highway freight vehicle and cargo based on an AI system includes steps 10 to 60, wherein:

[0066] Step 10: Acquire an initial vehicle image of a target freight vehicle passing through a preset highway section.

[0067] The AI-based method for detecting and warning the outer dimensions of a highway freight vehicle and cargo according to an embodiment of the present invention is applied to an AI-based system for detecting and warning the outer dimensions of a highway freight vehicle and cargo. The system includes a vehicle detection and warning center, a vehicle image acquisition device, an image sampling unit, an image conversion unit, an image reconstruction unit, a dimension detection unit, and a warning unit.

[0068] In an embodiment of the present invention, the vehicle detection and warning center is respectively connected to the vehicle image acquisition device, image sampling unit, image conversion unit, image reconstruction unit, size detection unit and warning unit to store and manage the data of each unit or device.

[0069] The vehicle image acquisition device can be installed on a predetermined highway section, such as an electronic camera bar or a speed detection bar on Highway Section A. This device can capture an initial vehicle image of a target freight vehicle traveling along the predetermined highway section. This initial vehicle image can be represented by HRs. Here, the initial vehicle image records the direction and intensity of light propagation in space, capturing depth information and rich visual details in the scene.

[0070] Step 20: downsample the initial vehicle image to obtain a first target vehicle image.

[0071] Step 30: Convert the first target vehicle image into a second target vehicle image.

[0072] Optionally, the initial vehicle image is downsampled to obtain a first target vehicle image, and the first target vehicle image is converted into a second target vehicle image. The first target vehicle image can be represented by LRs, and the second target vehicle image can be represented by LRM.

[0073] In one embodiment, an initial vehicle image with dimensions of 7×3×H×W is converted to a Y channel and then downsampled by a factor of s to convert it into a second target vehicle image in the horizontal direction only, with dimensions of 1×sH×7sW (7 represents 7 consecutive viewpoints on the same baseline, 3 represents the number of RGB channels, H and W represent the height and width of the high-resolution image, and s is the downsampling factor).

[0074] Step 40 : Input the central viewpoint image of the initial vehicle image and the second target vehicle image into an image reconstruction model to obtain a viewpoint image output by the image reconstruction model.

[0075] The image reconstruction model in this embodiment of the present invention includes a texture extractor layer, a disparity estimation network layer, a fusion upsampling layer, and a reverse mapping layer. The disparity estimation network layer is used to input the second target vehicle image into the disparity estimation network to obtain an initial disparity map output by the disparity estimation network. The texture extractor layer is used to input the central viewpoint image of the continuous initial vehicle image into the texture extractor to obtain a feature map containing high-resolution image texture information output by the texture extractor. The fusion upsampling layer is used to fuse and upsample the feature map and the initial disparity map to obtain a target disparity map. The reverse mapping layer is used to obtain a viewpoint image based on the target disparity map.

[0076] Specifically, after obtaining the second target vehicle image, the central viewpoint image HRc of the initial vehicle image and the second target vehicle image LRM may be input into the image reconstruction model to obtain the viewpoint image output by the image reconstruction model.

[0077] Among them, the image reconstruction model includes a texture extractor layer, a disparity estimation network layer, a fusion upsampling layer and a reverse mapping layer.

[0078] Here, the disparity estimation network layer is used to input the second target vehicle image into the disparity estimation network to obtain the initial disparity map output by the disparity estimation network. The 2D convolution module with a convolution kernel size of 3, a stride of 1, a void rate and a padding of (1, 7) is used to accurately extract the spatial feature information B×C×sH×7sW in the horizontal direction, set the D-layer disparity level, and use a 2D convolution module with a gradually increasing receptive field to extract the angular feature information and construct a high-dimensional feature volume B×160×D×sH×sW containing spatial and angular feature information (B is the batch number, C is the number of channels, 160 is the feature volume dimension, and D is the number of disparity layers). The 3D convolution module is used to convolve the feature volume containing spatial and angular feature information and reduce its dimensionality to B×V×D×sH×sW. Softmax is used to multiply the weights of each layer of the D dimension and then add them together to obtain the initial disparity map B×V×sH×sW (V is the number of initial disparity maps).

[0079] The texture extractor layer is used to input the central viewpoint image of the continuous initial vehicle image into the texture extractor, and the texture extractor outputs a feature map containing the texture information of the high-resolution image. Specifically, the texture extractor uses the initial convolution block, four layers of residual blocks, and a convolution block with a convolution kernel of 1 to extract the 1×H×W texture information of the Y channel of the central viewpoint high-resolution image.

[0080] The fusion upsampling layer is used to fuse and upsample the feature map and the initial disparity map to obtain the target disparity map. That is, the unfold function is used to fuse the extracted texture information with the initial disparity map, indirectly guiding the initial disparity map to be upsampled to the target disparity map, thereby achieving super-resolution reconstruction of the low-resolution disparity map.

[0081] The reverse mapping layer is used to generate viewpoint images based on the target disparity map. Using the target disparity map obtained in the previous steps, combined with the reverse mapping method, it is possible to map two sets of viewpoint images in the HRs, with the same virtual viewpoint position but different mapping directions between each pair of adjacent images. By cropping and fusing these two sets of images, 30 dense, high-quality viewpoint images are ultimately generated, achieving angular super-resolution reconstruction of sparse viewpoints.

[0082] Step 50 : Recognize the viewpoint image based on a preset vehicle and cargo outer dimension detection model to obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model; the vehicle and cargo outer dimension detection model is trained based on sample viewpoint images and their corresponding outer dimension labels.

[0083] Optionally, the viewpoint image is input into a preset vehicle and cargo outer dimension detection model, and the viewpoint image is recognized by the vehicle and cargo outer dimension detection model to obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model. The vehicle and cargo outer dimension detection model in the embodiment of the present invention is trained based on sample viewpoint images and their corresponding outer dimension labels.

[0084] Step 60: Issue an early warning based on the outer dimension detection information.

[0085] Optionally, an early warning is issued based on the external dimension detection information. Specifically, the external dimensions of the target freight vehicle and cargo are obtained based on the external dimension detection information, the external dimensions of the vehicle and cargo are compared with preset dimensions, and an early warning is issued based on the comparison result, wherein the preset dimensions are set according to actual conditions. In the event that the external dimensions of the vehicle and cargo exceed the preset dimensions, an early warning notification is sent to the user terminal. In another embodiment, at this time, the license plate number of the target freight vehicle can also be recorded, and the license plate number and the early warning notification are bound and sent to the user terminal. The process of obtaining the license plate number is not further described here. For example, it can be recognized by optical character recognition (OCR) methods, deep learning methods, template matching methods, etc.

[0086] The embodiment of the present invention performs resolution processing on the initial vehicle image to obtain a high-resolution viewpoint image, and then performs AI intelligent detection on the viewpoint image through a vehicle and cargo outer dimension detection model to obtain outer dimension detection information, and issues an early warning based on the outer dimension detection information. Therefore, an accurate early warning can be issued based on the artificial intelligence system combined with the high-resolution image, thereby improving the accuracy of vehicle and cargo outer dimension detection and early warning.

[0087] In one embodiment, the training steps of the image reconstruction model include: obtaining an initial sample vehicle image, downsampling the initial sample vehicle image to obtain a first target sample vehicle image, converting the first target sample vehicle image into a second target sample vehicle image, and determining the image model to be trained; inputting the central viewpoint image of the initial sample vehicle image and the second target sample vehicle image into the image model to be trained to obtain a predicted viewpoint image output by the image model to be trained; determining a target loss based on the difference between the initial sample vehicle image and the predicted viewpoint image, and performing parameter iteration on the image model to be trained based on the target loss to obtain an image reconstruction model.

[0088] Specifically, in order to better obtain the image reconstruction model and improve the model performance and the speed of angular super-resolution reconstruction, the image reconstruction model can be trained based on the following steps:

[0089] First, an initial sample vehicle image is obtained, the initial sample vehicle image is downsampled to obtain a first target sample vehicle image, the first target sample vehicle image is converted into a second target sample vehicle image, and an image model to be trained is determined.

[0090] Here, the parameters of the image model to be trained may be randomly generated or pre-set, and the embodiment of the present invention does not impose any specific limitation on this.

[0091] Then, the central viewpoint image of the initial sample vehicle image and the second target sample vehicle image can be input into the image model to be trained to obtain the predicted viewpoint image output by the image model to be trained.

[0092] After obtaining the predicted viewpoint image based on the image model to be trained, the target loss can be determined based on the difference between the initial sample vehicle image and the predicted viewpoint image, and based on the target loss, the parameters of the image model to be trained are iterated, and the image model to be trained after the parameter iteration is used as the image reconstruction model.

[0093] It can be understood that the greater the difference between the initial sample vehicle image and the predicted viewpoint image, the greater the target loss; and the smaller the difference between the initial sample vehicle image and the predicted viewpoint image, the smaller the target loss.

[0094] In one embodiment, the predicted viewpoint images include a plurality of first predicted viewpoint images and a plurality of second predicted viewpoint images.

[0095] Among them, multiple first predicted viewpoint images are obtained one by one through reverse mapping based on the target predicted disparity map and the initial sample vehicle image; multiple second predicted viewpoint images are obtained one by one through reverse mapping based on the target predicted disparity map and the initial sample vehicle image; the first predicted viewpoint image is a forward predicted viewpoint image, and the second predicted viewpoint image is a reverse predicted viewpoint image; the forward direction is mapped from left to right, and the reverse direction is mapped from right to left; the target predicted disparity map is the output of the fusion upsampling layer in the image model to be trained.

[0096] In one embodiment, a target loss is determined based on a difference between an initial sample vehicle image and a predicted viewpoint image, including: determining a first loss based on a difference between an initial sample vehicle image and a first predicted viewpoint image; determining a second loss based on a difference between an initial sample vehicle image and a second predicted viewpoint image; and determining a target loss based on the first loss and the second loss.

[0097] Specifically, the predicted viewpoint images include multiple first predicted viewpoint images and multiple second predicted viewpoint images. Here, the multiple first predicted viewpoint images are obtained one by one through reverse mapping based on the target predicted disparity map and the initial sample vehicle image. For example, through the high-resolution bidirectional disparity map predicted by the network, the first 6 viewpoints of the input 7 consecutive initial sample vehicle images HRs are sequentially mapped to the positions of the last 6 viewpoints to obtain the corresponding first predicted viewpoint image YL i (i=0,1,2,3,4,5,6).

[0098] Multiple second predicted viewpoint images are obtained by reverse mapping based on the target predicted disparity map and the initial sample vehicle image. For example, the high-resolution bidirectional disparity map predicted by the network is used to map the last 6 viewpoints of the input 7 consecutive initial sample vehicle images HRs to the positions of the first 6 viewpoints in sequence to obtain the corresponding second predicted viewpoint image YL i (i=0,1,2,3,4,5,6).

[0099] That is, the first predicted viewpoint image is a predicted viewpoint image in the forward direction (mapped from left to right), and the second predicted viewpoint image is a predicted viewpoint image in the reverse direction (mapped from right to left).

[0100] A first loss is determined based on a difference between an initial sample vehicle image and a first predicted viewpoint image, and a second loss is determined based on a difference between the initial sample vehicle image and a second predicted viewpoint image.

[0101] Furthermore, based on the first loss and the second loss, a target loss is determined.

[0102] In one embodiment, determining the target loss based on the first loss and the second loss includes: stacking the initial sample vehicle images in the height dimension to obtain a first stacked image; stacking the predicted viewpoint images in the height dimension to obtain a second stacked image; determining the stacking loss based on the difference between the first stacked image and the second stacked image; and determining the target loss based on the first loss, the second loss, and the stacking loss.

[0103] Specifically, the initial sample vehicle images are stacked in the height dimension (H dimension) to obtain a first stacked image, which is denoted as HRs[u:v]. The predicted viewpoint images are then stacked in the height dimension (H dimension) to obtain a second stacked image, which is denoted as Ys[u:v]. u and v represent the stacking of the u-th to v-1-th input high-resolution images in the H dimension.

[0104] The stack loss may be determined based on a difference between the first stack image and the second stack image.

[0105] Finally, the target loss is determined based on the first loss, the second loss, and the stacking loss. For example, by giving the target loss a larger weight in the early stages of training and gradually increasing the weight of the stacking loss in the later stages of training, the quality of the stacked virtual viewpoint images can be gradually improved until the overall loss function converges. Finally, during the inference phase, high-quality virtual novel viewpoint images are generated, while achieving an approximately fivefold increase in overall mapping speed.

[0106] This embodiment of the present invention proposes a stacked mapping loss based on the L1Loss loss function. This approach constrains the mean absolute error (MAE) of the newly mapped image generated by stacking the continuous images of the front portion of HRc on the vertical axis, and vice versa. By reducing the errors in these two sets of constraints, the target disparity map required for mapping can be continuously optimized, ensuring the quality of the virtual viewpoint image while improving the overall mapping speed.

[0107] In one embodiment, the texture extractor includes a multi-layer residual block and a first convolution layer, and the residual block sequentially includes a second convolution layer, a batch normalization layer, and an activation layer.

[0108] Specifically, considering that the texture features of high-resolution images are needed as guiding information during super-resolution reconstruction, and because the central viewpoint image of a set of continuous initial vehicle images contains the richest texture information and position information of the image main content and nearby objects, the Y channel data of the high-resolution central viewpoint image is input into the texture extractor.

[0109] The texture extractor includes multiple layers of residual blocks and a first convolutional layer with a convolution kernel of 1. The residual block sequentially includes a second convolutional layer, a batch normalization layer (BN), and an activation layer. The first and second convolutional layers can be cascaded multi-layer convolutional neural networks (CNNs), deep neural networks (DNNs), or a combination of CNNs and DNNs, etc., which are not specifically limited in this embodiment of the present invention. The activation layer here can use the LReLU activation function, the Softmax activation function, or the GELU (Gaussian Error Linear Unit) activation function, which are not specifically limited in this embodiment of the present invention.

[0110] After passing through the entire network, the output feature map has dimensions of B×128×sH×sW. This is followed by an upsampling module, which extracts a mask to highlight salient features while adjusting the feature map dimensions to the required B×36×sH×sW. The view function then adjusts the dimensions to (B, 1, 9, 2, 2, sH, sW). The unfold function then fuses the mask information with the low-resolution bidirectional disparity map, indirectly guiding upsampling to a high-resolution bidirectional disparity map of dimensions B×V×H×W. This module indirectly guides the upsampling of the low-resolution bidirectional disparity map to a high-resolution bidirectional disparity map.

[0111] In one embodiment, the disparity estimation network layer includes a horizontal spatial feature extraction module, a cost volume construction module, an aggregation module and a regression module; the horizontal spatial feature extraction module is used to extract spatial feature information in the horizontal direction of the second target vehicle image to obtain a first feature map; the cost volume construction module is used to obtain a high-dimensional feature volume based on the disparity level, void ratio, filling value, and the first feature map; the aggregation module is used to globally aggregate the high-dimensional feature volume to obtain a target aggregate; the regression module is used to regress the target aggregate to obtain an initial disparity map.

[0112] Specifically, the disparity estimation network layer includes a horizontal spatial feature extraction module, a cost volume construction module, an aggregation module, and a regression module.

[0113] The horizontal spatial feature extraction module extracts horizontal spatial feature information from the second target vehicle image, generating the first feature map. For example, the B×7×3×H×W initial vehicle image HRc is converted to YCRCB channels and then downsampled by a factor of 2. The Y channel data is then converted into a horizontally-only second target vehicle image of B×sH×7sW (B represents the batch size, 7 represents seven consecutive viewpoints on the same baseline, 3 represents the number of RGB channels, s represents the downsampling factor, and H and W represent the height and width of the high-resolution image). When processing a second target vehicle image with only horizontal pixel shift, the horizontal spatial feature extraction module accurately extracts horizontal spatial feature information from the low-resolution second target vehicle image.

[0114] The cost volume construction module is used to obtain a high-dimensional feature volume based on the disparity level, dilation rate, padding value, and the first feature map. For example, the dimensions of the first feature map after extracting spatial feature information are B×C×sH×7sW. In combination with the disparity layering method, 16 disparity levels are set. Each level uses a convolution with a gradually increasing dilation rate to improve the receptive field to extract feature information in the angular space. At the same time, in order to maintain the size of the feature map, the dilation rate dila and padding value pad are determined according to the formula to meet the following formula:

[0115] ;

[0116] ;

[0117] In equations (1) and (2), d is the preset disparity value, which is a non-negative integer, and A is the number of input viewpoints. In one embodiment, in the cost volume construction part, the convolution parameters are specifically set to a convolution kernel of 7, a stride, a dilation rate, and a padding of 1×7, 1×dila, and 3×pad, respectively. A high-dimensional feature volume B×160×16×sH×sW (160 is the number of channels, and 16 is the number of disparity level layers) containing angular and spatial feature information is constructed through 2D convolution layers with gradually increasing receptive fields.

[0118] Here, the aggregation module is used to globally aggregate the high-dimensional feature volumes to obtain the target aggregate. For example, the high-dimensional feature volumes are globally aggregated into the target aggregate B×V×16×sH×sW, where the global aggregation module is composed of multiple layers of 3D convolution and 3D convolution residual blocks.

[0119] The regression module regresses the target aggregate to obtain the initial disparity map. For example, the target aggregate is regressed by multiplying the weights of the 16 disparity layers using the Softmax function and then adding them together to produce the output low-resolution bidirectional disparity map B×V×sH×sW (V represents the number of bidirectional disparity maps required). In summary, the above four components enable fast estimation of low-resolution bidirectional disparity maps.

[0120] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the AI-based system-based method for detecting and warning the outer dimensions of a highway freight vehicle and cargo. The method includes:

[0121] Collecting an initial vehicle image of a target freight vehicle passing through a preset highway section;

[0122] Downsampling the initial vehicle image to obtain a first target vehicle image;

[0123] converting the first target vehicle image into a second target vehicle image;

[0124] Inputting the central viewpoint image of the initial vehicle image and the second target vehicle image into an image reconstruction model to obtain a viewpoint image output by the image reconstruction model;

[0125] Recognizing the viewpoint image based on a preset vehicle and cargo outer dimension detection model to obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model; the vehicle and cargo outer dimension detection model is trained based on sample viewpoint images and their corresponding outer dimension labels;

[0126] An early warning is issued based on the outer dimension detection information.

[0127] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0128] On the other hand, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for detecting and warning the outer dimensions of a highway freight vehicle and cargo based on an AI system provided by the above methods is implemented. The method includes:

[0129] Collecting an initial vehicle image of a target freight vehicle passing through a preset highway section;

[0130] Downsampling the initial vehicle image to obtain a first target vehicle image;

[0131] converting the first target vehicle image into a second target vehicle image;

[0132] Inputting the central viewpoint image of the initial vehicle image and the second target vehicle image into an image reconstruction model to obtain a viewpoint image output by the image reconstruction model;

[0133] Recognizing the viewpoint image based on a preset vehicle and cargo outer dimension detection model to obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model; the vehicle and cargo outer dimension detection model is trained based on sample viewpoint images and their corresponding outer dimension labels;

[0134] An early warning is issued based on the outer dimension detection information.

[0135] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program is implemented to perform the above-mentioned AI-based system-based highway freight vehicle and cargo outer dimensions detection and early warning method, the method comprising:

[0136] Collecting an initial vehicle image of a target freight vehicle passing through a preset highway section;

[0137] Downsampling the initial vehicle image to obtain a first target vehicle image;

[0138] converting the first target vehicle image into a second target vehicle image;

[0139] Inputting the central viewpoint image of the initial vehicle image and the second target vehicle image into an image reconstruction model to obtain a viewpoint image output by the image reconstruction model;

[0140] Recognizing the viewpoint image based on a preset vehicle and cargo outer dimension detection model to obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model; the vehicle and cargo outer dimension detection model is trained based on sample viewpoint images and their corresponding outer dimension labels;

[0141] An early warning is issued based on the outer dimension detection information.

[0142] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0143] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An AI-based road freight vehicle and cargo size detection and warning system, characterized by: It includes a vehicle detection and warning center, a vehicle image acquisition device, an image sampling unit, an image conversion unit, an image reconstruction unit, a size detection unit and an early warning unit; the vehicle detection and early warning center is connected to the vehicle image acquisition device, the image sampling unit, the image conversion unit, the image reconstruction unit, the size detection unit and the early warning unit respectively, and stores and manages the data of each unit or device; A vehicle image acquisition device is used to acquire an initial vehicle image of a target freight vehicle passing through a preset highway section; an image sampling unit, configured to downsample the initial vehicle image to obtain a first target vehicle image; an image conversion unit, configured to convert the first target vehicle image into a second target vehicle image; an image reconstruction unit, configured to input the central viewpoint image of the initial vehicle image and the second target vehicle image into an image reconstruction model to obtain a viewpoint image output by the image reconstruction model; a size detection unit, configured to recognize the viewpoint image based on a preset vehicle and cargo outer dimension detection model, and obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model; The vehicle and cargo outer dimension detection model is trained based on sample viewpoint images and their corresponding outer dimension labels; An early warning unit, configured to issue an early warning based on the outer dimension detection information; The training step of the image reconstruction model includes: obtaining an initial sample vehicle image, downsampling the initial sample vehicle image to obtain a first target sample vehicle image, converting the first target sample vehicle image into a second target sample vehicle image, and determining an image model to be trained; Inputting the central viewpoint image of the initial sample vehicle image and the second target sample vehicle image into the image model to be trained to obtain a predicted viewpoint image output by the image model to be trained; Determining a target loss based on a difference between the initial sample vehicle image and the predicted viewpoint image, and performing parameter iteration on the image model to be trained based on the target loss to obtain the image reconstruction model; The predicted viewpoint images include a plurality of first predicted viewpoint images and a plurality of second predicted viewpoint images; the plurality of first predicted viewpoint images are obtained by mapping the target predicted disparity map and the initial sample vehicle image one by one through reverse mapping; the plurality of second predicted viewpoint images are obtained by mapping the target predicted disparity map and the initial sample vehicle image one by one through reverse mapping; the first predicted viewpoint image is a forward predicted viewpoint image, and the second predicted viewpoint image is a reverse predicted viewpoint image; the forward direction is mapped from left to right, and the reverse direction is mapped from right to left; the target predicted disparity map is output by a fusion upsampling layer in the image model to be trained; Accordingly, determining the target loss based on the difference between the initial sample vehicle image and the predicted viewpoint image includes: determining a first loss based on a difference between the initial sample vehicle image and the first predicted viewpoint image; determining a second loss based on a difference between the initial sample vehicle image and the second predicted viewpoint image; determining the target loss based on the first loss and the second loss; The determining the target loss based on the first loss and the second loss includes: Stacking the initial sample vehicle images in a height dimension to obtain a first stacked image; stacking the predicted viewpoint images in a height dimension to obtain a second stacked image; determining a stack loss based on a difference between the first stack image and the second stack image; The target loss is determined based on the first loss, the second loss, and the stacking loss.

2. A method for detecting and warning the outer dimensions of a highway freight vehicle and cargo based on an AI system, applied to the system for detecting and warning the outer dimensions of a highway freight vehicle and cargo based on an AI system as claimed in claim 1, characterized in that: The AI-based method for detecting and warning the outer dimensions of a highway freight vehicle and its cargo includes: Collecting an initial vehicle image of a target freight vehicle passing through a preset highway section; Downsampling the initial vehicle image to obtain a first target vehicle image; converting the first target vehicle image into a second target vehicle image; Inputting the central viewpoint image of the initial vehicle image and the second target vehicle image into an image reconstruction model to obtain a viewpoint image output by the image reconstruction model; Recognizing the viewpoint image based on a preset vehicle and cargo outer dimension detection model to obtain outer dimension detection information output by the vehicle and cargo outer dimension detection model; the vehicle and cargo outer dimension detection model is trained based on sample viewpoint images and their corresponding outer dimension labels; Producing an early warning based on the outer dimension detection information; The image reconstruction model training step includes: Acquire an initial sample vehicle image, downsample the initial sample vehicle image to obtain a first target sample vehicle image, convert the first target sample vehicle image into a second target sample vehicle image, and determine an image model to be trained; Inputting the central viewpoint image of the initial sample vehicle image and the second target sample vehicle image into the image model to be trained to obtain a predicted viewpoint image output by the image model to be trained; Determining a target loss based on a difference between the initial sample vehicle image and the predicted viewpoint image, and performing parameter iteration on the image model to be trained based on the target loss to obtain the image reconstruction model; The predicted viewpoint images include a plurality of first predicted viewpoint images and a plurality of second predicted viewpoint images; the plurality of first predicted viewpoint images are obtained by mapping the target predicted disparity map and the initial sample vehicle image one by one through reverse mapping; the plurality of second predicted viewpoint images are obtained by mapping the target predicted disparity map and the initial sample vehicle image one by one through reverse mapping; the first predicted viewpoint image is a forward predicted viewpoint image, and the second predicted viewpoint image is a reverse predicted viewpoint image; the forward direction is mapped from left to right, and the reverse direction is mapped from right to left; the target predicted disparity map is output by a fusion upsampling layer in the image model to be trained; Accordingly, determining the target loss based on the difference between the initial sample vehicle image and the predicted viewpoint image includes: determining a first loss based on a difference between the initial sample vehicle image and the first predicted viewpoint image; determining a second loss based on a difference between the initial sample vehicle image and the second predicted viewpoint image; determining the target loss based on the first loss and the second loss; The determining the target loss based on the first loss and the second loss includes: Stacking the initial sample vehicle images in a height dimension to obtain a first stacked image; stacking the predicted viewpoint images in a height dimension to obtain a second stacked image; determining a stack loss based on a difference between the first stack image and the second stack image; The target loss is determined based on the first loss, the second loss, and the stacking loss.

3. The AI-based method for detecting and warning the outer dimensions of freight vehicles and cargo according to claim 2 is characterized in that: The image reconstruction model includes a texture extractor layer, a disparity estimation network layer, a fusion upsampling layer and a reverse mapping layer; The disparity estimation network layer is used to input the second target vehicle image into the disparity estimation network to obtain an initial disparity map output by the disparity estimation network; The texture extractor layer is used to input the central viewpoint images of the continuous initial vehicle images into the texture extractor, and obtain the feature map containing high-resolution image texture information output by the texture extractor; The fusion upsampling layer is used to fuse and upsample the feature map and the initial disparity map to obtain a target disparity map; The reverse mapping layer is used to obtain the viewpoint image based on the target disparity map.

4. The AI-based method for detecting and warning the outer dimensions of freight vehicles and cargo according to claim 3 is characterized in that: The texture extractor includes a multi-layer residual block and a first convolution layer, and the residual block sequentially includes a second convolution layer, a batch normalization layer and an activation layer.

5. The AI-based method for detecting and warning the outer dimensions of freight vehicles and cargo according to claim 3 is characterized in that: The disparity estimation network layer includes a horizontal space feature extraction module, a cost volume construction module, an aggregation module and a regression module; The horizontal spatial feature extraction module is used to extract spatial feature information in the horizontal direction of the second target vehicle image to obtain a first feature map; The cost volume construction module is used to obtain a high-dimensional feature volume based on the disparity level, the hole rate, the filling value, and the first feature map; The aggregation module is used to perform global aggregation on the high-dimensional feature volume to obtain a target aggregate; The regression module is used to regress the target aggregate to obtain the initial disparity map.

6. An electronic device comprising: The memory and the processor are characterized in that a computer software program is stored on the memory, and when the processor reads and executes the computer software program, the method for detecting and warning the outer dimensions of a highway freight vehicle and cargo based on an AI system as described in any one of claims 2 to 5 is implemented.

7. A non-transitory computer-readable storage medium, characterized in that A computer software program is stored, and when the computer software program is executed by a processor, the method for detecting and warning the outer dimensions of a highway freight vehicle and cargo based on an AI system as described in any one of claims 2 to 5 is implemented.

Images