Bicycle collision risk early warning method and system based on AI and multi-dimensional sensing

Abstract

The invention relates to the technical field of bicycle collision early warning, and discloses a bicycle collision risk early warning method and system based on AI and multi-dimensional sensing, and the method comprises the steps: obtaining a target bicycle, training a semantic annotation device through employing a training image data set, obtaining an image annotation device, carrying out the road shooting based on a portable camera in the target bicycle, and obtaining an image annotation device; the method comprises the steps of obtaining an original road image, carrying out pixel clustering on the original road image to obtain a pixel segmentation image, labeling the pixel segmentation image by using an image labeling device and the original road image to obtain a labeled road image, and carrying out collision prediction by using a binocular road image and the labeled road image to obtain collision time. And performing road risk assessment based on the marked road image to obtain a road risk degree, and executing pre-constructed early warning operation according to the collision time and the road risk degree. According to the invention, the accuracy of danger early warning during the driving of the bicycle can be improved, and the intelligent degree of early warning is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bicycle collision warning, in particular to a bicycle collision risk warning method and system based on AI and multi-dimensional sensing. BACKGROUND

[0002] With the surge in urban cycling, bicycles often collide with cars and pedestrians when mixed with them due to the lack of active warning means, causing personal injury and traffic congestion, so a real-time collision risk warning method that can be implemented on ordinary bicycles is needed to perceive and resolve potential collisions in advance.

[0003] The traditional method runs a general target detection network in the rear end of a monocular camera and combines a fixed threshold alarm, which can alert the front obstacles, but has the defects of high false alarm rate, inability to accurately estimate the collision time, and sharp decline in detection accuracy in night or strong light environment. SUMMARY

[0004] The present application provides a bicycle collision risk warning method and system based on AI and multi-dimensional sensing, which mainly aims to improve the accuracy of danger warning when the bicycle is driving and improve the intelligent degree of warning.

[0005] To achieve the above purpose, the present application provides a bicycle collision risk warning method based on AI and multi-dimensional sensing, which comprises: Obtaining a target bicycle, wherein the target bicycle is equipped with a portable camera and a sensor group, wherein the sensor group comprises a binocular vision sensor; Training a pre-constructed semantic labeler using a pre-constructed training image dataset to obtain an image labeler; Capturing a road using the portable camera in the target bicycle to obtain an original road image, performing pixel clustering on the original road image to obtain a pixel segmentation image, wherein the pixel segmentation image comprises a plurality of pixel blocks, and each pixel block is composed of a plurality of segmented pixel points; Labeling the pixel segmentation image using the image labeler and the original road image to obtain a labeled road image, collecting a binocular road image using the sensor group, and predicting a collision using the binocular road image and the labeled road image to obtain a collision time; Evaluating the road danger based on the labeled road image to obtain a road danger degree, and performing a pre-constructed warning operation according to the collision time and the road danger degree.

[0006] Optionally, the training of the pre-constructed semantic labeler using the pre-constructed training image dataset to obtain the image labeler comprises: The training image data in the training image data set is sequentially extracted, and training road images and semantic labels in the training image data are confirmed, wherein the training road image is an RGB image; The training road image is input into the semantic labeler to obtain a reconstructed road image and a plurality of semantic segmentation images, wherein the semantic labeler comprises a variational autoencoder, an RGB decoder, and a plurality of semantic decoders, and each semantic decoder corresponds to one semantic label, wherein the reconstructed road image corresponds to the RGB decoder in the semantic labeler, and the semantic segmentation image corresponds to one of the semantic decoders in the semantic labeler; A plurality of true labels of the plurality of semantic segmentation images are obtained based on the semantic labels, wherein the true label corresponds to the semantic segmentation image; A training loss value is calculated according to the training road image, the reconstructed road image, the plurality of semantic segmentation images, and the plurality of true labels, and it is determined whether the training loss value is less than a preset loss threshold; If the training loss value is not less than the loss threshold, the variational autoencoder, the RGB decoder, and the plurality of semantic decoders in the semantic labeler are updated based on the training loss value to obtain an updated labeler; The updated labeler is used as the semantic labeler, and the step of sequentially extracting the training image data in the training image data set is returned until the training loss value is less than the loss threshold; If the training loss value is less than the loss threshold, the semantic labeler is recorded as an image labeler.

[0007] Optionally, the step of inputting the training road image into the semantic labeler to obtain the reconstructed road image and the plurality of semantic segmentation images comprises: The training road image is input into the variational autoencoder in the semantic labeler to obtain a mean tensor and a standard deviation tensor, wherein the tensor shape of the mean tensor and the tensor shape of the standard deviation tensor are the same; Resampling is performed based on the mean tensor and the standard deviation tensor to obtain a resampling parameter tensor; The resampling parameter tensor is segmented according to a preset number of semantic categories to obtain a reconstruction parameter tensor and a plurality of semantic probability tensors, wherein the number of semantic probability tensors is the same as the number of semantic categories, and the tensor shape of the semantic probability tensor is the same as the tensor shape of the reconstruction parameter tensor; The reconstruction parameter tensor is input into the RGB decoder to obtain the reconstructed road image, and the plurality of semantic probability tensors are input into the plurality of semantic decoders respectively to obtain the plurality of semantic segmentation images, wherein the pixel points in the reconstructed road image represent RGB values, and the pixel points in the semantic segmentation image represent probability values, wherein the semantic probability tensor corresponds to the semantic decoder.

[0008] Optionally, the labeling the pixel segmentation image by using the image labeler and the original road image to obtain a labeled road image comprises: inputting the original road image into the image labeler to obtain a plurality of current semantic probability maps, wherein each current semantic probability map corresponds to a semantic label; extracting pixel blocks in the pixel segmentation image in sequence, and performing semantic labeling on the pixel blocks according to the plurality of current semantic probability maps to obtain labeled pixel blocks; collecting the labeled pixel blocks to obtain a plurality of labeled pixel blocks, and updating the pixel segmentation image by using the plurality of labeled pixel blocks to obtain the labeled road image.

[0009] Optionally, the performing semantic labeling on the pixel blocks according to the plurality of current semantic probability maps to obtain labeled pixel blocks comprises: performing the following operations on each segmented pixel point in the pixel block: confirming an original position of the segmented pixel point in the original road image; identifying a plurality of same-position pixel points in the plurality of current semantic probability maps based on the original position, and identifying a plurality of same-position pixel values corresponding to the plurality of same-position pixel points, wherein the same-position pixel values are probability values; confirming a maximum pixel value in the plurality of same-position pixel values, and recording a current semantic probability map corresponding to the maximum pixel value as a target semantic probability map, and recording a semantic label corresponding to the target semantic probability map as a target label; collecting the target labels corresponding to each segmented pixel point in the pixel block to obtain a target label set, counting a majority label in the target label set, and recording the majority label as a pixel block label; labeling the pixel block by using the pixel block label to obtain a labeled pixel block.

[0010] Optionally, the performing collision prediction by using the binocular road image and the labeled road image to obtain a collision time comprises: constructing a disparity map according to a preset stereo matching algorithm and the binocular road image; extracting labeled pixel blocks in the labeled road image in sequence, and confirming a center pixel point of the labeled pixel block; identifying a disparity pixel point corresponding to the center pixel point in the disparity map, and obtaining a disparity pixel value of the disparity pixel point; if the disparity pixel value is not greater than a preset disparity threshold, recording the labeled pixel block as a collision pixel block; collecting the collision pixel blocks to obtain a plurality of collision pixel blocks, and identifying a plurality of collision disparity regions in the disparity map based on the plurality of collision pixel blocks, wherein the collision disparity regions correspond to the collision pixel blocks in a one-to-one manner; performing collision prediction based on the plurality of collision disparity regions to obtain a collision time.

[0011] Optionally, the collision prediction based on the plurality of collision disparity regions comprises: extracting collision disparity regions in the plurality of collision disparity regions in sequence, and confirming a set of collision pixel points in the collision disparity regions; calculating a disparity mean value based on the set of collision pixel points, and calculating a current observation distance according to the disparity mean value and preset camera parameters, wherein the camera parameters comprise camera intrinsic parameters and camera extrinsic parameters; detecting an observation relative speed of the collision disparity region; calculating an original collision time according to the current observation distance, the observation relative speed, and a preset reaction time length, wherein the original collision time is represented as: wherein, the original collision time is represented as, the reaction time length is represented as, the observation relative speed is represented as, a preset bicycle acceleration is represented as, the current observation distance is represented as; summing up the original collision times to obtain a plurality of original collision times, and recording a minimum value in the plurality of original collision times as the collision time.

[0012] Optionally, the detection of the observation relative speed of the collision disparity region comprises: confirming a current collection time, and obtaining a front collection time based on the current collection time and a preset collection interval; detecting a front disparity map based on the front collection time; identifying a front disparity region corresponding to the collision disparity region in the front disparity map, calculating a front disparity mean value of the front disparity region, and calculating a front observation distance according to the front disparity mean value and the camera parameters; calculating the observation relative speed according to the front observation distance, the current observation distance, and the collection interval.

[0013] Optionally, the road danger assessment based on the labeled road image comprises: identifying a set of road object categories in the labeled road image, wherein the set of road object categories comprises a plurality of road object categories, and the road object categories comprise vehicles, pedestrians, and stationary obstacles, wherein the road object categories correspond to semantic labels; respectively counting the number of objects of each road object category in the set of road object categories based on the labeled road image to obtain a set of object numbers, and calculating a road danger degree according to the set of object numbers, wherein the road danger degree is represented as: wherein, represents a road danger degree, represents a number of road object categories in a road object category set, represents a weight of a th road object category in a preset road object category set, represents a th object quantity in an object quantity set, represents an exponential function with a natural constant as a base.

[0014] To achieve the above object, the application further provides a bicycle collision risk early warning system based on AI and multi-dimensional sensing, comprising: A labeler construction module is configured to obtain a target bicycle, wherein the target bicycle is equipped with a portable camera and a sensor group, wherein the sensor group comprises a binocular vision sensor, a pre-constructed training image dataset is used to train a pre-constructed semantic labeler, and an image labeler is obtained. A pixel image segmentation module is configured to capture a road based on the portable camera in the target bicycle, obtain an original road image, and perform pixel clustering on the original road image to obtain a pixel segmentation image, wherein the pixel segmentation image comprises a plurality of pixel blocks, and each pixel block is composed of a plurality of segmented pixel points. A collision time prediction module is configured to label the pixel segmentation image using the image labeler and the original road image to obtain a labeled road image, collect a binocular road image using the sensor group, and perform collision prediction using the binocular road image and the labeled road image to obtain a collision time. A road danger degree evaluation module is configured to evaluate road danger based on the labeled road image to obtain a road danger degree, and perform a pre-constructed early warning operation according to the collision time and the road danger degree.

[0015] To solve the above problems, the application further provides an electronic device, comprising: A memory is configured to store at least one instruction; A processor is configured to execute the instruction stored in the memory to implement the above-mentioned bicycle collision risk early warning method based on AI and multi-dimensional sensing.

[0016] To solve the above problems, the application further provides a computer readable storage medium, wherein the computer readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned bicycle collision risk early warning method based on AI and multi-dimensional sensing.

[0017] The present application is to solve the problems described in the background art, first, the pre-constructed semantic labeler is trained using the training image dataset to obtain an image labeler, this step uses a variational autoencoder to train RGB reconstruction and multi semantic decoding in parallel, thereby learning the appearance features and three types of semantics of vehicles, pedestrians and stationary obstacles at the same time, thereby avoiding the delay of multi-model cascade, and through the reconstruction loss, overfitting is inhibited, so that the subsequent labeling still maintains high precision under low-quality images such as night and glare, then the original road image is pixel clustered to obtain a pixel segmentation image, this step uses SLIC superpixel pre-segmentation to compress the million-pixel-level image into a hundred-pixel block, and then does semantic voting at the block level instead of the pixel level, which not only greatly reduces the amount of calculation, but also retains the object edges, so that the embedded device can also run the subsequent depth estimation and collision prediction in real time, further, the pixel segmentation image is labeled using the image labeler and the original road image to obtain a marked road image, this step does majority label fusion within the superpixel block, which can quickly smooth single-pixel noise, even if a decoder has low confidence, it can also output stable semantics through block-level consensus, significantly reducing the false detection rate and ensuring the reliability of the subsequent collision area screening, then multiple collision disparity regions are identified using the binocular road image and the marked road image, collision prediction is performed based on the multiple collision disparity regions to obtain a collision time, this step directly embeds semantic labels into the disparity map, only calculates the mean disparity and relative speed for key blocks such as vehicles and pedestrians, and eliminates invalid backgrounds, finally, road danger is evaluated based on the road object category set to obtain a road danger degree, and the pre-constructed warning operation is executed according to the collision time and the road danger degree, this step converts the vehicle density and the number of pedestrians into a unified danger indicator, and then jointly decides with the collision time, automatically triggers the heat feeling and vibration in the daytime noisy environment, and only uses flashing prompts on the night silent road section, avoiding excessive disturbance to cyclists, thereby making the warning operation of the present scheme highly intelligent. Therefore, the present application can improve the accuracy of the danger warning of the bicycle during driving and improve the intelligent degree of the warning. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The flowchart of the bicycle collision risk warning method based on AI and multi-dimensional sensing provided by an embodiment of the present application is shown; Figure 2 The functional module diagram of the bicycle collision risk warning system based on AI and multi-dimensional sensing provided by an embodiment of the present application is shown; Figure 3 The structural diagram of the electronic device for implementing the bicycle collision risk warning method based on AI and multi-dimensional sensing provided by an embodiment of the present application is shown.

[0019] Explanation of reference signs: 1, electronic device; 10, processor; 11, memory; 12, bus.

[0020] The objectives, functional characteristics and advantages of the present application will be further explained in conjunction with the embodiments, with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein merely exemplify the application and do not limit the application.

[0022] Embodiments of the present application provide a bicycle collision risk warning method based on AI and multi-dimensional sensing. The execution subject of the bicycle collision risk warning method based on AI and multi-dimensional sensing includes but is not limited to at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiments of the present application. In other words, the bicycle collision risk warning method based on AI and multi-dimensional sensing can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server, or a cloud server cluster, etc.

[0023] Reference Figure 1 As shown in the figure, the flowchart of the bicycle collision risk warning method based on AI and multi-dimensional sensing provided by an embodiment of the present application. In this embodiment, the bicycle collision risk warning method based on AI and multi-dimensional sensing includes: S1, acquiring a target bicycle, wherein the target bicycle is equipped with a portable camera and a sensor group, wherein the sensor group includes a binocular vision sensor.

[0024] It can be understood that the target bicycle refers to a bicycle that needs to be warned of collision risk. The portable camera refers to a lightweight image acquisition device, for example, a motion camera equipped with a CMOS sensor, which is installed at the handlebar or front fork of the bicycle and faces the front direction. The binocular vision sensor refers to a stereo vision system composed of two parallel arranged cameras, for example, Intel RealSense D435 depth camera. In addition to the binocular vision sensor, the above-mentioned sensor group can also include a speed sensor, an acceleration sensor, a gyroscope, a GPS positioning device, etc.

[0025] S2, training a pre-constructed semantic labeler using a pre-constructed training image dataset to obtain an image labeler.

[0026] The training image dataset refers to a set of multiple training image data, wherein the training image data is a combination of training road images and semantic labels. The training road images refer to road images collected by relevant personnel at different times and on different roads, and the training road images are collected by a portable camera mounted on a bicycle. After obtaining the training road images, the objects appearing in the training road images need to be labeled by humans. These labels are semantic labels, and these semantic labels include vehicles, pedestrians, stationary obstacles, and empty labels. Each pixel point in the training road image corresponds to a semantic label, wherein the empty label refers to a label that does not belong to a vehicle, a pedestrian, or a stationary obstacle. For example, in a training road image, a car occupies the middle area of the image, and the pixels of the car are labeled as "vehicle". The pedestrian on the right side of the image is labeled as "pedestrian", the garbage can on the roadside is labeled as "stationary obstacle", and the background area such as the sky and the road surface is labeled as "empty label".

[0027] Further, the semantic labeler refers to a structure composed of an RGB decoder, a variational autoencoder, and multiple semantic decoders. The RGB decoder refers to the decoder branch of the variational autoencoder (VAE) responsible for image reconstruction. The role of the RGB decoder is to decode the latent feature vector back to an RGB image with the same size as the input image, which is used to supervise the integrity of feature learning. The semantic decoder refers to the decoder branch of the VAE responsible for semantic segmentation. The role of the semantic decoder is to output a probability map corresponding to each semantic class, indicating the confidence of each pixel belonging to the class. Each semantic decoder corresponds to a semantic label, for example, there are three semantic decoders: semantic decoder A1, semantic decoder A2, and semantic decoder A3, which correspond to a semantic label respectively. Semantic decoder A1 corresponds to "vehicle", semantic decoder A2 corresponds to "pedestrian", and semantic decoder A3 corresponds to "stationary obstacle". The structure of the above RGB decoder is composed of multiple deconvolution layers and up-sampling layers, and finally connects a 3-channel convolution layer (used to output R, G, and B values respectively). The RGB decoder can output a reconstructed RGB image (i.e. the subsequent reconstructed road image). The structure of the above semantic decoder is similar to that of the RGB decoder, but finally connects a 1-channel convolution layer (used to output probability values). The semantic decoder can output a single-channel semantic probability map (i.e. the subsequent semantic segmentation image). The image labeler refers to the trained semantic labeler. The image labeler can perform semantic labeling on each pixel point in the input image, i.e. output multiple images, and the pixel value of each pixel point in each image represents a probability value.

[0028] In detail, the training of the pre-constructed semantic labeler using the pre-constructed training image dataset to obtain the image labeler includes: Training image data is extracted sequentially from the training image dataset to identify the training road images and semantic annotations in the training image data. The training road images are RGB images. The training road image is input into the semantic labeler to obtain the reconstructed road image and multiple semantic segmentation images. The semantic labeler includes a variational autoencoder, an RGB decoder and multiple semantic decoders, and each semantic decoder corresponds to a semantic label. The reconstructed road image corresponds to the RGB decoder in the semantic labeler, and the semantic segmentation image corresponds one-to-one with the semantic decoder in the semantic labeler. Multiple ground truth annotations are obtained from the multiple semantic segmentation images based on semantic annotation, wherein each ground truth annotation corresponds one-to-one with a semantic segmentation image; The training loss value is calculated based on the training road image, the reconstructed road image, multiple semantic segmentation images, and multiple ground truth annotations. It is then determined whether the training loss value is less than a preset loss threshold. If the training loss value is not less than the loss threshold, the variational autoencoder, RGB decoder and multiple semantic decoders in the semantic annotator are updated based on the training loss value to obtain the updated annotator; Update the labeler as the semantic labeler and return to the step of sequentially extracting training image data from the training image dataset until the training loss value is less than the loss threshold. If the training loss value is less than the loss threshold, then the semantic annotator is recorded as an image annotator.

[0029] It can be understood that the reconstructed road image refers to an image output by the training road image, and a pixel value corresponding to a pixel point in the reconstructed road image is an RGB color value. The reconstructed road image is used to measure whether the encoder effectively learns the overall structure and content information of the image, and a reconstruction loss is calculated through the reconstructed road image. The reconstruction loss can be used to constrain the model to avoid overfitting. The semantic segmentation image refers to an image output by the semantic labeler. A pixel value corresponding to a pixel point in the semantic segmentation image is a probability value. The probability value represents a probability that the pixel point belongs to a semantic label corresponding to the semantic labeler. The semantic segmentation image is used to compare with the real label to calculate a segmentation accuracy loss (i.e., a semantic segmentation loss used to calculate the loss value subsequently). The real label refers to a set of semantic labels of each pixel point in the semantic segmentation image. The training loss value refers to a scalar value that measures the difference between the prediction result of the model and the real label. The specific way of calculating the training loss value according to the training road image, the reconstructed road image, the plurality of semantic segmentation images, and the plurality of real labels is as follows: a reconstruction loss (such as mean square error, MSE) is calculated through the reconstructed road image and the training road image, and a semantic segmentation loss (such as cross-entropy loss) is calculated through the plurality of semantic segmentation images and the plurality of real labels. The reconstruction loss is used to ensure the image reconstruction quality, and the semantic segmentation loss is used to improve the semantic labeling accuracy. The reconstruction loss and the semantic segmentation loss are weighted and summed as a total loss, which is the training loss value. The loss threshold refers to a loss convergence threshold set artificially. When the training loss value is greater than the loss threshold, it indicates that the semantic labeler has not sufficiently learned the characteristics of the training data, i.e., training needs to be continued at this time. The updating of the variational autoencoder, the RGB decoder, and the semantic decoder in the semantic labeler based on the training loss value refers to calculating the gradient of the loss with respect to the model parameters using the backpropagation algorithm, and updating the network weights using an optimizer (such as Adam) to gradually reduce the loss value. This process is a prior art and will not be described here. The updated labeler refers to the semantic labeler after being updated.

[0030] In detail, the inputting of the training road image into the semantic labeler to obtain the reconstructed road image and the plurality of semantic segmentation images comprises: The inputting of the training road image into the variational autoencoder in the semantic labeler to obtain the mean tensor and the standard deviation tensor, wherein the tensor shape of the mean tensor and the tensor shape of the standard deviation tensor are the same; The resampling based on the mean tensor and the standard deviation tensor to obtain the resampling parameter tensor; The segmentation of the resampling parameter tensor according to the preset number of semantic categories to obtain the reconstruction parameter tensor and the plurality of semantic probability tensors, wherein the number of the semantic probability tensors is the same as the number of the semantic categories, and the tensor shape of the semantic probability tensor is the same as the tensor shape of the reconstruction parameter tensor; The reconstruction parameter tensor is input into the RGB decoder to obtain a reconstructed road image, and the plurality of semantic probability tensors are input into the plurality of semantic decoders to obtain a plurality of semantic segmentation images, wherein a pixel point in the reconstructed road image represents an RGB value, and a pixel point in the semantic segmentation image represents a probability value, and the semantic probability tensor corresponds to the semantic decoder one by one.

[0031] It can be understood that the mean tensor refers to a tensor composed of the expected value (i.e., the mean) of the Gaussian distribution estimated by the encoder network of the variational autoencoder for each feature point in the latent space after feature extraction of the input image. The standard deviation tensor refers to a tensor composed of the standard deviation of the Gaussian distribution estimated by the encoder network for each feature point in the same latent space. The latent space, the expected value of the Gaussian distribution, and the standard deviation of the Gaussian distribution are all existing concepts and technologies in the variational autoencoder, and will not be described here. The resampling parameter tensor refers to a tensor of latent variables sampled from the mean tensor and the standard deviation tensor, wherein resampling based on the mean tensor and the standard deviation tensor refers to generating a latent variable using the reparameterization trick, and the latent variable is the resampling parameter tensor. This step is a prior art and will not be described here. The number of semantic categories refers to the number of semantic labels, i.e., the number of semantic decoders, for example: the semantic labels include vehicles, pedestrians, and stationary obstacles, and the number of semantic categories is 3. The reconstruction parameter tensor refers to part of the latent variables used to input the RGB decoder, and the semantic probability tensor refers to part of the latent variables used to input each semantic decoder, wherein the resampling parameter tensor is divided according to the predetermined number of semantic categories, i.e., the resampling parameter tensor is uniformly divided into a plurality of (one more than the number of semantic categories) sub-tensors along the channel dimension (i.e., the reconstruction parameter tensor and the plurality of semantic probability tensors), for example: if the resampling parameter tensor has a shape of [16, 16, 256], the number of semantic categories is 3, and it is divided into 4 parts (1 part for reconstruction and 3 parts for semantic decoding), and the number of channels of each part is 64, i.e., the shape of each sub-tensor is [16, 16, 64].

[0032] S3, based on a portable camera in the target bicycle, a road is shot to obtain an original road image, and pixel clustering is performed on the original road image to obtain a pixel segmentation image, wherein the pixel segmentation image includes a plurality of pixel blocks, and each pixel block is composed of a plurality of segmented pixel points.

[0033] It can be understood that the original road image refers to an image obtained after shooting by a portable camera, and the image is an RGB image. The pixel segmentation image refers to the original road image after pixel clustering, and the pixel segmentation image includes a plurality of pixel blocks, each pixel block represents an image region with similar color and spatial features, and a plurality of segmented pixel points in each pixel block have high consistency in color and position. The purpose of pixel clustering of the original road image is to aggregate pixel points with similar color and spatial position in the image into a plurality of superpixel blocks. The clustering method is to use a superpixel segmentation algorithm (such as the SLIC algorithm) to perform iterative clustering based on the CIELAB color space and coordinate position of the pixel.

[0034] S4, label the pixel segmentation image using the image labeler and the original road image to obtain a labeled road image, collect a binocular road image using a sensor group, and perform collision prediction using the binocular road image and the labeled road image to obtain a collision time.

[0035] It needs to be explained that the labeled road image refers to a labeled image, and the labeled road image includes a plurality of labeled pixel blocks, and each labeled pixel block corresponds to a semantic label. These labeled pixel blocks correspond to the pixel blocks in the pixel segmentation image, and after labeling, the specific objects represented by these labeled pixel blocks can be clearly determined, for example, the semantic label of a certain labeled pixel block is "vehicle", and the object represented by the labeled pixel block in the original road image is a vehicle. The binocular road image refers to an image collected by a binocular vision sensor in the sensor group, and the binocular road image includes a right road image and a left road image, and the right road image and the left road image come from a right camera and a left camera in the binocular vision sensor, respectively. The collision time refers to the time required for the target bicycle to collide.

[0036] In detail, the labeling of the pixel segmentation image using the image labeler and the original road image to obtain the labeled road image includes: inputting the original road image into the image labeler to obtain a plurality of current semantic probability maps, wherein each current semantic probability map corresponds to a semantic label; extracting pixel blocks in the pixel segmentation image in sequence, and performing semantic labeling on the pixel blocks according to the plurality of current semantic probability maps to obtain labeled pixel blocks; collecting the labeled pixel blocks to obtain a plurality of labeled pixel blocks, and updating the pixel segmentation image using the plurality of labeled pixel blocks to obtain the labeled road image.

[0037] It can be understood that the plurality of current semantic probability maps refer to a plurality of images containing probability values output by a plurality of semantic decoders in the image labeler. The labeled pixel block refers to a pixel block that has been semantically labeled. The updating of the pixel segmentation image using the plurality of labeled pixel blocks refers to replacing a plurality of pixel blocks in the pixel segmentation image with a plurality of labeled pixel blocks, and the pixel segmentation image after the replacement is the labeled road image. The labeled road image labels objects such as vehicles and pedestrians.

[0038] In detail, the semantic labeling of the pixel block according to the plurality of current semantic probability maps to obtain the labeled pixel block comprises: The following operations are performed on each segmented pixel point in the pixel block: Confirming the original position of the segmented pixel point in the original road image; Identifying a plurality of co-located pixel points in the plurality of current semantic probability maps based on the original position, and identifying a plurality of co-located pixel values corresponding to the plurality of co-located pixel points, wherein the co-located pixel values are probability values; Confirming the maximum pixel value in the plurality of co-located pixel values, and recording the current semantic probability map corresponding to the maximum pixel value as a target semantic probability map, and recording the semantic label corresponding to the target semantic probability map as a target label; Summarizing the target labels corresponding to each segmented pixel point in the pixel block to obtain a target label set, counting the most frequent label in the target label set, and recording the most frequent label as the pixel block label; Labeling the pixel block using the pixel block label to obtain the labeled pixel block.

[0039] It can be understood that the original position refers to the coordinates of the segmented pixel point in the original road image. The co-located pixel point refers to a pixel point in the current semantic probability map that has the same position as the original position. The co-located pixel value refers to the pixel value of the co-located pixel point, which is a probability value indicating the probability that the segmented pixel point belongs to the semantic label corresponding to the current semantic probability map. The maximum pixel value refers to the maximum value in the plurality of co-located pixel values. The target semantic probability map refers to the current semantic probability map corresponding to the maximum pixel value. The most frequent label refers to the target label that appears most frequently in the target label set.

[0040] In detail, the collision prediction using the binocular road image and the labeled road image to obtain the collision time comprises: Constructing a disparity map according to a preset stereo matching algorithm and the binocular road image; Extracting the labeled pixel block in the labeled road image in sequence, and confirming the center pixel point of the labeled pixel block; Identifying the disparity pixel point corresponding to the center pixel point in the disparity map, and obtaining the disparity pixel value of the disparity pixel point; If the parallax pixel value is not greater than the preset parallax threshold value, the marked pixel block is recorded as a collision pixel block; The collision pixel blocks are summarized to obtain a plurality of collision pixel blocks, and a plurality of collision parallax regions are identified in the parallax map based on the plurality of collision pixel blocks, wherein the collision parallax regions correspond to the collision pixel blocks one by one. Collision prediction is performed based on the plurality of collision parallax regions to obtain a collision time.

[0041] It can be understood that the stereo matching algorithm refers to an algorithm for calculating the parallax of corresponding points of left and right images (i.e., right and left road images), such as a semi-global matching algorithm. The parallax map refers to a gray-scale image of the same size as the marked road image, which represents the horizontal displacement difference of each pixel point in the right and left road images. The construction method of the parallax map is a prior art and will not be described here. The center pixel point refers to the geometric center of the marked pixel block. The parallax pixel point refers to the pixel point corresponding to the center pixel point in the parallax map. The parallax pixel point is obtained by first obtaining the position of the center pixel point in the marked road image, which is recorded as the center position, and then determining the pixel point corresponding to the center position in the parallax map, which is the parallax pixel point. The parallax pixel value refers to the pixel value of the parallax pixel point, which represents the actual distance corresponding to the pixel point. The smaller the parallax, the closer the distance. The parallax threshold value is a human-set distance warning value. When the parallax pixel value is greater than the parallax threshold value, it indicates that the marked pixel block corresponding to the parallax pixel value is far away from the target bicycle and there is no risk of collision in a short time. If the parallax pixel value is not greater than the parallax threshold value, it indicates that the marked pixel block is close to the target bicycle and there is a risk of collision. Therefore, the marked pixel block at this time is recorded as a collision pixel block. The collision parallax region represents the region corresponding to the object that may collide in the marked road image.

[0042] In detail, the collision prediction based on the plurality of collision parallax regions to obtain a collision time comprises: collision parallax regions are extracted in the plurality of collision parallax regions, and a set of collision pixel points in the collision parallax region is confirmed; a parallax mean value is calculated based on the set of collision pixel points, and a current observation distance is calculated according to the parallax mean value and preset camera parameters, wherein the camera parameters include camera intrinsic parameters and camera extrinsic parameters; an observation relative speed of the collision parallax region is detected; an original collision time is calculated according to the current observation distance, the observation relative speed, and a preset reaction time length, wherein the original collision time is represented as: wherein, represents the original collision time, denotes a reaction time length, denotes an observed relative speed, denotes a preset bicycle acceleration, denotes a current observed distance; aggregate the original collision time, obtain a plurality of original collision times, and record a minimum value in the plurality of original collision times as the collision time.

[0043] It can be understood that the disparity mean refers to an average value of pixel values corresponding to all collision pixel points in the collision pixel point set. The camera parameter refers to a geometric parameter of the binocular vision sensor, and the camera parameter includes camera intrinsic parameters and camera extrinsic parameters. The camera intrinsic parameters are, for example, focal length, principal point coordinates, etc., and the camera extrinsic parameters are, for example, baseline length, camera inter-rotation translation matrix, etc. The bicycle acceleration refers to the acceleration of the current bicycle.

[0044] Further, the current observed distance refers to an actual distance between the target bicycle and an object (for example, a vehicle, a pedestrian, etc.) corresponding to the collision disparity region. The calculation formula of the current observed distance is: wherein, denotes a baseline length, denotes a focal length, denotes a disparity mean. The observed relative speed refers to a relative speed between the target bicycle and the object corresponding to the collision disparity region. In the calculation formula of the collision time, denotes a time required for the target bicycle to decelerate at the bicycle acceleration from the deceleration to the stop of the bicycle. The original collision time refers to a time length of the collision between the target bicycle and the collision disparity region. The reaction time length refers to a time length of the reaction of the rider to the subsequent warning operation.

[0045] In detail, the detection of the observed relative speed of the collision disparity region comprises: confirming a current collection time, obtaining a front collection time based on the current collection time and a preset collection interval; detecting a front disparity map based on the front collection time; identifying a front disparity region corresponding to the collision disparity region in the front disparity map, calculating a front disparity mean of the front disparity region, and calculating a front observed distance according to the front disparity mean and the camera parameter; calculating the observed relative speed according to the front observed distance, the current observed distance, and the collection interval.

[0046] It can be understood that the current acquisition time point refers to the time point at which the current observation distance is obtained. The acquisition interval refers to the time interval during which the binocular vision sensor performs image acquisition. The previous acquisition time point refers to the time point at which the binocular vision sensor last performed image acquisition from the current acquisition time point. The previous disparity map refers to the disparity map obtained at the previous acquisition time point. The previous disparity region refers to the region corresponding to the collision disparity region in the previous disparity map, and the previous disparity region is obtained in the following manner: the geometric center of the collision disparity region is determined, the geometric center is recorded as a collision center, the current position of the collision center in the disparity map is recorded, the pixel point at the current position in the previous disparity map is queried, the pixel point is recorded as a previous collision center, the previous marking pixel block in which the previous collision center is located (the previous marking pixel block refers to the marking pixel block in the marking road image obtained at the previous acquisition time point) is determined, and the previous marking pixel block is recorded as the previous disparity region. The previous disparity mean value refers to the disparity mean value of the previous disparity region. The previous observation distance refers to the observation distance corresponding to the previous disparity mean value. The calculation method of the observation relative speed is: (current observation distance-previous observation distance) / acquisition interval.

[0047] S5, performing road danger assessment based on the marking road image to obtain a road danger degree, and performing a pre-constructed warning operation according to the collision time and the road danger degree.

[0048] It should be explained that the road danger degree refers to a numerical value quantifying the danger degree of the current road (i.e., the road on which the target bicycle is currently traveling), and the more vehicles, pedestrians, and stationary obstacles on the current road, the greater the danger degree of the road. The warning operation refers to one or more of visual warning, auditory warning, and tactile warning, wherein the visual warning refers to red flashing of the screen installed on the target bicycle, the auditory warning refers to the alarm emitted by the buzzer, and the tactile warning refers to the vibration of the handlebar of the target bicycle.

[0049] In detail, the road danger assessment based on the marking road image to obtain the road danger degree includes: identifying a road object class set in the marking road image, wherein the road object class set includes a plurality of road object classes, and the road object class includes: a vehicle, a pedestrian, and a stationary obstacle, wherein the road object class corresponds to a semantic label; based on the marking road image, respectively counting the number of objects of each road object class in the road object class set to obtain an object number set, and calculating the road danger degree according to the object number set, wherein the road danger degree is represented as: wherein, represents the road danger degree, represents the number of road object classes in the road object class set, a weight of an i-th road object category in a preset road object category set, a weight of an i-th road object category in a preset road object category set, an i-th object quantity in an object quantity set, an i-th object quantity in an object quantity set, represents an exponential function with a natural constant as a base.

[0050] It can be understood that the road object category set refers to a collection of multiple road object categories. The object quantity refers to the number of road object categories appearing in the marked road image, for example, if there are 3 marked pixel blocks marked as "vehicle" in the marked road image, the object quantity corresponding to the road object category "vehicle" is 3. The weight of the road object category refers to a numerical value quantifying the dangerous degree of the road object category. The greater the weight, the more dangerous the corresponding road object category, and for example, a vehicle is more dangerous than a pedestrian and a stationary obstacle, so the weights corresponding to the vehicle, the pedestrian and the stationary obstacle can be set as 0.5, 0.3 and 0.2 respectively.

[0051] It should be explained that the generation of the multi-modal warning instruction according to the collision time and the road danger degree means that when the collision time is greater than a collision threshold, the collision threshold refers to a time threshold set by a person, which means that there is a potential collision risk at this time, and a warning is needed. Visual warning or auditory warning can be selected as the warning operation, for example, when riding at night, visual warning is more obvious, and visual warning can be selected, and when riding in the daytime, auditory warning can be selected. At the same time, if the road danger degree is greater than a danger degree threshold, the danger degree threshold refers to a danger degree threshold set by a person, which means that the situation on the road at this time is more complex, for example, the noise is larger, the light is brighter, etc., and simple auditory warning or tactile warning may not be received by the rider, therefore, at this time, visual warning, auditory warning and tactile warning need to be performed at the same time, so as to ensure that the warning information can be effectively conveyed to the rider in different environments.

[0052] In order to solve the problems in the background art, first, the pre-constructed semantic labeler is trained by using the training image data set to obtain an image labeler, this step uses the variational autoencoder to train the RGB reconstruction and multi semantic decoding in parallel, thereby learning the appearance features and the vehicle, pedestrian and static obstacle three types of semantics at the same time, thereby avoiding the delay of multi-model cascade, and the overfitting is inhibited through the reconstruction loss, so that the subsequent labeling still maintains high precision under low-quality images such as night and glare, then the original road image is pixel clustered to obtain a pixel segmentation image, this step adopts SLIC superpixel pre-segmentation to compress the million-pixel-level image into a hundred-pixel block, and then the semantic voting is performed at the block level instead of the pixel level, which not only greatly reduces the calculation amount, but also retains the object edge, so that the embedded device can also run the subsequent depth estimation and collision prediction in real time, further, the pixel segmentation image is labeled by using the image labeler and the original road image to obtain a marked road image, this step fuses the crowd value label inside the superpixel block, which can quickly smooth the single-pixel noise, and even if a decoder has low confidence, the stable semantics can be output through the block-level consensus, thereby significantly reducing the false detection rate and ensuring the reliability of the subsequent collision area screening, then a plurality of collision disparity regions are identified by using the binocular road image and the marked road image, collision prediction is performed based on the plurality of collision disparity regions to obtain a collision time, this step directly embeds the semantic label into the disparity map, only calculates the mean disparity and relative speed for the key blocks such as vehicles and pedestrians, and eliminates invalid backgrounds, finally, road danger is evaluated based on the road object category set to obtain a road danger degree, and the pre-constructed warning operation is performed according to the collision time and the road danger degree, this step converts the vehicle density and the number of pedestrians into a unified danger index, and then jointly decides with the collision time, so as to automatically trigger the heat feeling and vibration in the daytime noisy environment, and only use the flashing prompt in the night silent section, thereby avoiding excessive disturbance to the rider, and further making the warning operation of the present scheme have high intelligence.

[0053] As Figure 2 shown, it is a functional module diagram of a bicycle collision risk warning system based on AI and multi-dimensional sensing provided by an embodiment of the present application.

[0054] The bicycle collision risk warning system based on AI and multi-dimensional sensing 100 can be installed in an electronic device. According to the functions to be realized, the bicycle collision risk warning system based on AI and multi-dimensional sensing 100 can include a labeler construction module 101, a pixel image segmentation module 102, a collision time prediction module 103 and a road danger degree evaluation module 104. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, which are stored in the memory of the electronic device.

[0055] The marker construction module 101 is configured to obtain a target bicycle, wherein the target bicycle is equipped with a portable camera and a sensor group, wherein the sensor group comprises a binocular vision sensor, a pre-constructed semantic marker is trained by using a pre-constructed training image dataset, and an image marker is obtained. The pixel image segmentation module 102 is configured to capture a road by using the portable camera in the target bicycle to obtain an original road image, perform pixel clustering on the original road image, and obtain a pixel segmentation image, wherein the pixel segmentation image comprises a plurality of pixel blocks, and each pixel block is composed of a plurality of segmented pixel points. The collision time prediction module 103 is configured to label the pixel segmentation image by using the image marker and the original road image to obtain a marked road image, collect a binocular road image by using the sensor group, and perform collision prediction by using the binocular road image and the marked road image to obtain a collision time. The road danger degree evaluation module 104 is configured to evaluate road danger based on the marked road image to obtain a road danger degree, and perform a pre-constructed warning operation according to the collision time and the road danger degree.

[0056] In detail, the modules in the bicycle collision risk warning system 100 based on AI and multi-dimensional sensing in the embodiment of the present application use the same technical means as the bicycle collision risk warning method based on AI and multi-dimensional sensing in the above Figure 1 , and can produce the same technical effects, which will not be described here.

[0057] As shown in Figure 3 , it is a structural schematic diagram of an electronic device for implementing the bicycle collision risk warning method based on AI and multi-dimensional sensing according to an embodiment of the present application.

[0058] The electronic device 1 can include a processor 10, a memory 11 and a bus 12, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as a bicycle collision risk warning method program based on AI and multi-dimensional sensing.

[0059] The memory 11 includes at least one type of readable storage medium, such as flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 1. Further, the memory 11 includes both an internal storage unit and an external storage device of the electronic device 1. The memory 11 can be used not only to store application software and various data installed on the electronic device 1, such as the code of the bicycle collision risk early warning method program based on AI and multi-dimensional sensing, but also to temporarily store data that has been output or will be output.

[0060] The processor 10 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips, etc. The processor 10 is the control core of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, executes programs or modules stored in the memory 11 (such as the bicycle collision risk early warning method program based on AI and multi-dimensional sensing, etc.), and calls data stored in the memory 11 to perform various functions and process data of the electronic device 1.

[0061] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0062] Figure 3 Only an electronic device with components is shown, and those skilled in the art can understand that, Figure 3The illustrated structure does not constitute a limitation on the electronic device 1, and can include fewer or more components than illustrated, or combine certain components, or different component arrangements.

[0063] For example, although not shown, the electronic device 1 can also include a power source (such as a battery) to power the various components. Preferably, the power source can be logically connected to the at least one processor 10 through a power management system, so that functions such as charge management, discharge management, and power consumption management can be achieved through the power management system. The power source can also include one or more DC or AC power sources, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and any other components. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0064] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device 1 and other electronic devices.

[0065] Optionally, the electronic device 1 can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device 1 and to display a visualized user interface.

[0066] The program of the bicycle collision risk early warning method based on AI and multi-dimensional sensing stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, which, when executed in the processor 10, can achieve: obtaining a target bicycle, wherein the target bicycle is equipped with a portable camera and a sensor group, wherein the sensor group includes a binocular vision sensor; training a pre-constructed semantic labeler using a pre-constructed training image dataset to obtain an image labeler; capturing a road using the portable camera in the target bicycle to obtain an original road image, performing pixel clustering on the original road image to obtain a pixel segmentation image, wherein the pixel segmentation image includes a plurality of pixel blocks, and each pixel block is composed of a plurality of segmented pixel points; The pixel segmentation image is labeled by using the image labeler and the original road image to obtain a marked road image, the binocular road image is collected by using the sensor group, and collision prediction is performed by using the binocular road image and the marked road image to obtain a collision time. Road danger is evaluated based on the marked road image to obtain a road danger degree, and a pre-constructed warning operation is executed according to the collision time and the road danger degree.

[0067] Specifically, the specific implementation method of the processor 10 to the above instructions can refer to Figures 1 to 3 The description of related steps in the corresponding embodiment will not be repeated here.

[0068] Further, the modules / units integrated in the electronic device 1 are stored in a computer readable storage medium if they are realized in the form of software function units and sold or used as independent products. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or system capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory).

[0069] The application also provides a computer readable storage medium, the readable storage medium stores a computer program, the computer program can realize the following when being executed by the processor of the electronic device: Obtain a target bicycle, wherein the target bicycle is equipped with a portable camera and a sensor group, wherein the sensor group includes a binocular vision sensor; The pre-constructed semantic labeler is trained by using a pre-constructed training image data set to obtain an image labeler; Roads are photographed by using the portable camera in the target bicycle to obtain original road images, and the original road images are subjected to pixel clustering to obtain pixel segmentation images, wherein the pixel segmentation images include a plurality of pixel blocks, and each pixel block is composed of a plurality of segmented pixel points; The pixel segmentation image is labeled by using the image labeler and the original road image to obtain a marked road image, the binocular road image is collected by using the sensor group, and collision prediction is performed by using the binocular road image and the marked road image to obtain a collision time; Road danger is evaluated based on the marked road image to obtain a road danger degree, and a pre-constructed warning operation is executed according to the collision time and the road danger degree.

[0070] In the several embodiments of the application, it should be understood that the disclosed device, system and method can be implemented in other ways. For example, the system embodiments described above are only schematic. Actual implementation can have another division way.

[0071] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0072] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0073] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for early warning of bicycle collision risk based on AI and multi-dimensional sensing, characterized in that, The method includes: Acquire the target bicycle, which is equipped with a portable camera and a sensor group, including a binocular vision sensor; The pre-built semantic annotator is trained using a pre-built training image dataset to obtain an image annotator; The road is captured by a portable camera in the target bicycle to obtain the original road image. The original road image is then clustered into pixels to obtain a pixel segmentation image. The pixel segmentation image includes multiple pixel blocks, and each pixel block is composed of multiple segmentation pixels. The pixel segmentation image is labeled using an image annotator and the original road image to obtain a labeled road image. A stereo road image is acquired using a sensor array. Collision prediction is performed using the stereo road image and the labeled road image to obtain the collision time. Road hazard assessment is performed based on marked road images to obtain road hazard level, and pre-built early warning operations are executed based on collision time and road hazard level.

2. The bicycle collision risk warning method based on AI and multi-dimensional sensing as described in claim 1, characterized in that, The step of training a pre-built semantic annotator using a pre-built training image dataset to obtain an image annotator includes: Training image data is extracted sequentially from the training image dataset to identify the training road images and semantic annotations in the training image data. The training road images are RGB images. The training road image is input into the semantic labeler to obtain the reconstructed road image and multiple semantic segmentation images. The semantic labeler includes a variational autoencoder, an RGB decoder and multiple semantic decoders, and each semantic decoder corresponds to a semantic label. The reconstructed road image corresponds to the RGB decoder in the semantic labeler, and the semantic segmentation image corresponds one-to-one with the semantic decoder in the semantic labeler. Multiple ground truth annotations are obtained from the multiple semantic segmentation images based on semantic annotation, wherein each ground truth annotation corresponds one-to-one with a semantic segmentation image; The training loss value is calculated based on the training road image, the reconstructed road image, multiple semantic segmentation images, and multiple ground truth annotations. It is then determined whether the training loss value is less than a preset loss threshold. If the training loss value is not less than the loss threshold, the variational autoencoder, RGB decoder and multiple semantic decoders in the semantic annotator are updated based on the training loss value to obtain the updated annotator; Update the labeler as the semantic labeler and return to the step of sequentially extracting training image data from the training image dataset until the training loss value is less than the loss threshold. If the training loss value is less than the loss threshold, then the semantic annotator is recorded as an image annotator.

3. The bicycle collision risk warning method based on AI and multi-dimensional sensing as described in claim 2, characterized in that, The step of inputting the training road image into the semantic labeler to obtain the reconstructed road image and multiple semantic segmentation images includes: The training road image is input into the variational autoencoder in the semantic labeler to obtain the mean tensor and the standard deviation tensor. The mean tensor and the standard deviation tensor have the same tensor shape. Resampling is performed based on the mean tensor and the standard deviation tensor to obtain the resampling parameter tensor; The resampling parameter tensor is segmented according to the preset number of semantic categories to obtain the reconstruction parameter tensor and multiple semantic probability tensors. The number of semantic probability tensors is the same as the number of semantic categories, and the tensor shape of the semantic probability tensor is the same as the tensor shape of the reconstruction parameter tensor. The reconstruction parameter tensor is input into the RGB decoder to obtain the reconstructed road image. Multiple semantic probability tensors are input into multiple semantic decoders to obtain multiple semantic segmentation images. In the reconstructed road image, the pixels represent RGB values, and the pixels in the semantic segmentation image represent probability values. The semantic probability tensor corresponds one-to-one with the semantic decoder.

4. The bicycle collision risk warning method based on AI and multi-dimensional sensing as described in claim 3, characterized in that, The step of annotating the pixel-segmented image using an image annotator and the original road image to obtain a marked road image includes: The original road image is input into the image labeler to obtain multiple current semantic probability maps, each of which corresponds to a semantic label; In the pixel segmentation image, pixel blocks are extracted sequentially, and semantic labels are applied to the pixel blocks based on multiple current semantic probability maps to obtain labeled pixel blocks; The labeled pixel blocks are aggregated to obtain multiple labeled pixel blocks. These multiple labeled pixel blocks are then used to update the pixel segmentation image, resulting in a labeled road image.

5. The bicycle collision risk warning method based on AI and multi-dimensional sensing as described in claim 4, characterized in that, The step of semantically labeling pixel blocks based on multiple current semantic probability maps to obtain labeled pixel blocks includes: For each segmented pixel in the pixel block, the following operation is performed: The original positions of the segmented pixels in the original road image were determined. Based on the original location, identify multiple pixels at the same position in multiple current semantic probability maps, and identify multiple pixel values ​​at the same position corresponding to the multiple pixels at the same position, where the pixel values ​​at the same position are probability values; The maximum pixel value among multiple pixel values ​​at the same position is identified, and the current semantic probability map corresponding to the maximum pixel value is recorded as the target semantic probability map. The semantic label corresponding to the target semantic probability map is recorded as the target label. The target labels corresponding to each segmented pixel in the pixel block are summarized to obtain the target label set. The most common labels in the target label set are counted and recorded as the pixel block labels. The pixel blocks are marked using pixel block labels to obtain marked pixel blocks.

6. The bicycle collision risk warning method based on AI and multi-dimensional sensing as described in claim 5, characterized in that, The method of using binocular road images and marked road images to predict collision times includes: A disparity map is constructed based on a preset stereo matching algorithm and binocular road images; Extract the marked pixel blocks sequentially from the marked road image and identify the center pixel of each marked pixel block; In the disparity map, identify the disparity pixel corresponding to the center pixel and obtain the disparity pixel value of the disparity pixel. If the disparity pixel value is not greater than the preset disparity threshold, then the marked pixel block is recorded as a collision pixel block; By summing up the collision pixel blocks, multiple collision pixel blocks are obtained. Based on these multiple collision pixel blocks, multiple collision disparity regions are identified in the disparity map. Each collision disparity region corresponds one-to-one with a collision pixel block. Collision prediction is performed based on multiple collision parallax regions to obtain the collision time.

7. The bicycle collision risk warning method based on AI and multi-dimensional sensing as described in claim 6, characterized in that, The collision prediction based on multiple collision disparity regions, to obtain the collision time, includes: The collision parallax regions are extracted sequentially from multiple collision parallax regions to identify the collision pixel set within each region. The disparity mean is calculated based on the collision pixel set, and the current observation distance is calculated based on the disparity mean and preset camera parameters, where the camera parameters include: camera intrinsic parameters and camera extrinsic parameters. Detect the observed relative velocity in the collision parallax region; The initial collision time is calculated based on the current observation distance, the relative observation velocity, and the preset reaction time. The initial collision time is expressed as: in, Indicates the original collision time. Indicates reaction time. Indicates the observed relative velocity, This represents the preset bicycle acceleration. Indicates the current observation distance; Summarize the original collision times to obtain multiple original collision times, and record the minimum value among the multiple original collision times as the collision time.

8. The bicycle collision risk warning method based on AI and multi-dimensional sensing as described in claim 7, characterized in that, The detection of the observed relative velocity in the collision parallax region includes: Once the current acquisition time is determined, the previous acquisition time is obtained based on the current acquisition time and the preset acquisition interval. Detect the front disparity map based on the previous acquisition time; Identify the foreground parallax region corresponding to the collision parallax region in the foreground parallax map, calculate the foreground parallax mean of the foreground parallax region, and calculate the foreground observation distance based on the foreground parallax mean and camera parameters; The relative velocity of observation is calculated based on the previous observation distance, the current observation distance, and the acquisition interval.

9. The bicycle collision risk warning method based on AI and multi-dimensional sensing as described in claim 8, characterized in that, The road hazard assessment based on marked road images, to obtain the road hazard level, includes: Identify a set of road object categories in a labeled road image, wherein the set of road object categories includes multiple road object categories, and the road object categories include: vehicles, pedestrians and stationary obstacles, wherein the road object categories correspond to semantic labels; Based on the marked road images, the number of objects in each road object category is counted in the road object category set to obtain the object count set. The road hazard is then calculated based on the object count set, where the road hazard is expressed as: in, Indicates the degree of road hazard. This indicates the number of road object categories in the road object category set. This indicates the first in the preset road object category set. The weight of each road object category, Represents the first in the set of object quantities Number of objects This represents an exponential function with the natural constant as its base.

10. A bicycle collision risk warning system based on AI and multi-dimensional sensing, characterized in that, The system includes: The annotation construction module is used to acquire the target bicycle, which is equipped with a portable camera and a sensor group, including a binocular vision sensor. The pre-built semantic annotation is trained using a pre-built training image dataset to obtain an image annotation. The pixel image segmentation module is used to capture road images based on the portable camera in the target bicycle, obtain the original road image, and perform pixel clustering on the original road image to obtain a pixel segmentation image. The pixel segmentation image includes multiple pixel blocks, and each pixel block is composed of multiple segmentation pixels. The collision time prediction module is used to annotate the pixel segmented image using an image annotator and the original road image to obtain a marked road image, to acquire binocular road images using a sensor group, and to perform collision prediction using the binocular road images and the marked road images to obtain the collision time. The road hazard assessment module is used to assess road hazard based on marked road images, obtain road hazard level, and execute pre-built early warning operations based on collision time and road hazard level.

Images