Weather recognition method, device and equipment based on roadside sensing equipment

Abstract

The present invention relates to the field of intelligent transportation technology, and discloses a weather recognition method, device, and equipment based on roadside sensing equipment. The method includes: obtaining historical image data collected by the roadside sensing equipment; inputting the historical image data into a pre-trained multimodal model to determine sample slices and the correspondence between the roadside sensing equipment and the highly sensitive area; training the weather recognition model with the sample slices as a training data set to obtain a trained weather recognition model; obtaining image data to be tested; and determining weather forecast results based on the image data to be tested and the correspondence using the trained weather recognition model. The solution of the present invention determines sample slices and trains the weather recognition model using a pre-trained multimodal model, so that the trained weather recognition model pays more attention to the highly sensitive area, and because the corresponding area has explicit characteristics, the model reduces the demand for computing resources while ensuring the accuracy of the prediction results.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a weather recognition method, device, and equipment based on roadside sensing equipment. Background Art

[0002] With the rapid development of computer vision technology, image classification has been widely used in many fields. Especially in weather recognition, judging weather conditions by analyzing image data has become a hot research topic. Traditional weather recognition methods mainly rely on meteorological sensors and meteorological models. Although these methods are accurate, they are costly and have high requirements on environmental conditions.

[0003] In recent years, deep learning technology, especially convolutional neural networks (CNNs), has shown excellent performance in image classification tasks. Using CNNs for image classification has achieved remarkable results in fields such as object recognition and face recognition. However, applying this technology to weather recognition still faces some challenges, such as how to effectively deal with the impact of various lighting and weather conditions on image quality, and how to deal with the diversity of weather features in images.

[0004] In the related art, weather recognition methods mainly rely on multimodal data fusion and complex feature extraction technology, which are relatively complex to implement and have high requirements for computing resources. Therefore, there is a corresponding demand in the related art for recognition with low computing resource requirements and accurate weather forecasting. Summary of the invention

[0005] In view of this, the present invention provides a weather identification method, device, and equipment based on roadside sensing equipment to solve the technical problem of high demand for computing resources in related technologies.

[0006] In a first aspect, the present invention provides a weather recognition method based on a roadside sensing device, the method comprising: obtaining historical image data collected by the roadside sensing device; inputting the historical image data into a pre-trained multimodal model to determine sample slices and the correspondence between the roadside sensing device and the highly sensitive area; training the weather recognition model using the sample slices as a training data set to obtain a trained weather recognition model; obtaining the image data to be tested collected by the roadside sensing device; and determining the weather forecast result based on the image data to be tested and the correspondence using the trained weather recognition model.

[0007] In combination with the first aspect, in a possible implementation of the first aspect, historical image data is input into a pre-trained multimodal model to determine sample slices, including: inputting historical image data into a pre-trained multimodal model to determine a characteristic thermal map corresponding to a roadside sensing device; based on the characteristic thermal map, marking the locations of high thermal areas corresponding to specific weather types to form sample slices corresponding to the locations of the high thermal areas.

[0008] In combination with the first aspect, in a possible implementation of the first aspect, historical image data is input into a pre-trained multimodal model to determine the correspondence between roadside perception devices and highly sensitive areas, including: inputting historical image data into a pre-trained multimodal model to determine a characteristic heat map corresponding to the roadside perception device; based on the characteristic heat map, marking the location of the high thermal area corresponding to the specific weather type, thereby forming a correspondence between the roadside perception device and the highly sensitive area.

[0009] In combination with the first aspect, in a possible implementation of the first aspect, the training process of the weather recognition model includes: using sample slices as training data sets and inputting the weather recognition model; after the weather recognition model focuses on the location of the high thermal area corresponding to the specific weather type, completing the training of the weather recognition model to obtain a trained weather recognition model.

[0010] In combination with the first aspect, in a possible implementation method of the first aspect, based on the image data to be tested and the corresponding relationship, a weather forecast result is determined using a trained weather recognition model, including: based on the corresponding relationship, determining the location information of the highly sensitive area corresponding to each roadside sensing device; based on the location information and the image data to be tested, determining the weather forecast result using the trained weather recognition model.

[0011] In combination with the first aspect, in a possible implementation of the first aspect, the method also includes: receiving meteorological information and / or road information; based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, reminding the vehicle.

[0012] In combination with the first aspect, in a possible implementation method of the first aspect, based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, the vehicle is reminded, including: based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, the vehicle is reminded of information through roadside equipment; or, based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, information reminders are sent to the vehicle using subscription information.

[0013] In combination with the first aspect, in a possible implementation method of the first aspect, based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, the vehicle is reminded, including: based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, sending reminder information to the roadside communication module of the roadside sensing device; based on the roadside communication module interacting with the vehicle's on-board unit, providing information reminders to the vehicle.

[0014] In a second aspect, the present invention provides a weather recognition device based on a roadside sensing device, the device comprising: an image data acquisition module, used to acquire image data corresponding to the roadside sensing device; a sample slice determination module, used to input historical image data of the image data into a pre-trained multimodal model, and determine the correspondence between sample slices and the roadside sensing device and the highly sensitive area; a training module, used to train a weather recognition model using sample slices as a training data set to obtain a trained weather recognition model; and a result prediction module, used to determine a weather prediction result based on the image data and the correspondence using the trained weather recognition model.

[0015] In combination with the second aspect, in a possible implementation of the second aspect, the device also includes: an information receiving module for receiving meteorological information and / or road information; a vehicle reminder module for reminding the vehicle based on the weather forecast results and the received meteorological information and / or road information combined with the vehicle's trajectory information.

[0016] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the weather identification method based on a roadside sensing device according to the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the weather identification method based on a roadside sensing device according to the first aspect or any corresponding embodiment thereof.

[0018] The technical solution of the present invention has the following advantages:

[0019] The present invention provides a method, device and equipment for weather recognition based on roadside sensing equipment. The method uses historical image data collected by the roadside sensing equipment and uses the historical image data to input a pre-trained multimodal model to determine sample slices and the correspondence between the roadside sensing equipment and the highly sensitive area, thereby using the sample slices to train the weather recognition model, and using the correspondence between the roadside sensing equipment and the highly sensitive area, the weather forecast result is determined by completing the trained weather recognition model. In this process, by determining the sample slices, the data in the sample slices have a highly sensitive area that can better reflect the weather characteristics compared to the original data, and the sample slices are used to train the weather recognition model, so that the trained weather recognition model pays more attention to the highly sensitive area, grasps the corresponding dominant features, and reduces the amount of data calculation when applying the model, that is, when the real-time collected image to be tested is input to complete the trained weather recognition model, the model only needs to pay attention to the highly sensitive area, and because the corresponding area has dominant features, the model reduces the demand for computing resources while ensuring the accuracy of the prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a flowchart of a weather identification method based on a roadside sensing device provided according to an embodiment of the present invention;

[0022] Figure 2 is a structural block diagram of a weather recognition device based on a roadside sensing device provided according to an embodiment of the present invention;

[0023] Figure 3 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] According to an embodiment of the present invention, a weather identification method embodiment based on a roadside sensing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] This embodiment provides a weather recognition method based on roadside sensing equipment, such as Figure 1 As shown, the method comprises the following steps:

[0027] S101. Acquire historical image data collected by roadside sensing equipment.

[0028] Specifically, the roadside perception equipment includes: cameras, laser radars, signal collectors and other equipment installed on the road side. The image data corresponding to the roadside perception equipment refers to the image data collected by roadside perception equipment such as cameras, laser radars, signal collectors and other equipment installed on the road side, including information such as timestamps. Among them, historical image data refers to pre-collected image data, and the historical image data is used as training data. The historical image data is collected by one or more roadside perception equipment, that is, each roadside perception equipment has corresponding historical image data, which is the data basis for determining the corresponding relationship between the roadside perception equipment and the highly sensitive area in the subsequent determination.

[0029] S102: Input historical image data into a pre-trained multimodal model to determine the correspondence between sample slices and roadside perception devices and highly sensitive areas.

[0030] Specifically, inputting historical image data into a pre-trained multimodal model to determine the correspondence between sample slices and roadside perception devices and highly sensitive areas means inputting historical image data into a pre-trained multimodal model to determine the characteristic thermal map corresponding to the roadside perception device, and determining the location of the high thermal area corresponding to the specific weather type through the marked target box, thereby determining the highly sensitive area and the correspondence between the roadside perception device and the highly sensitive area.

[0031] S103: training a weather recognition model using the sample slices as a training data set to obtain a trained weather recognition model.

[0032] Specifically, the weather recognition model is trained using sample slices as training data sets. Obtaining a trained weather recognition model means inputting sample slices as training data sets into the weather recognition model until the weather recognition model stops converging, thereby obtaining a trained weather recognition model. The trained weather recognition model pays more attention to highly sensitive areas, masters corresponding explicit features, and provides a data basis for reducing the amount of data calculation in subsequent model applications.

[0033] S104: Acquire image data to be tested collected by a roadside sensing device.

[0034] Specifically, the image data to be tested refers to image data collected in real time. Similarly, the image data to be tested is collected by one or more roadside perception devices, that is, each roadside perception device has corresponding image data to be tested.

[0035] S105: Based on the image data to be tested and the corresponding relationship, the trained weather recognition model is used to determine the weather forecast result.

[0036] Specifically, based on the corresponding relationship between the image data to be tested and using the trained weather recognition model, determining the weather forecast result means determining the location information of the highly sensitive area corresponding to each roadside sensing device through the corresponding relationship between the roadside sensing device and the highly sensitive area, and using the trained weather recognition model to predict the corresponding position in the image data to be tested.

[0037] The present invention provides a weather recognition method, device and equipment based on roadside sensing equipment. The method uses historical image data collected by the roadside sensing equipment and uses the historical image data to input a pre-trained multimodal model to determine sample slices and the correspondence between the roadside sensing equipment and the highly sensitive area, so as to train the weather recognition model using the sample slices, and use the correspondence between the roadside sensing equipment and the highly sensitive area to complete the trained weather recognition model to determine the weather forecast result. In this process, by determining the sample slices, the data in the sample slices have a highly sensitive area that can better reflect the weather characteristics compared with the original data, and the sample slices are used to train the weather recognition model, so that the trained weather recognition model pays more attention to the highly sensitive area, grasps the corresponding dominant features, and reduces the amount of data calculation when applying the model, that is, when the real-time collected image to be tested is input to complete the trained weather recognition model, the model only needs to pay attention to the highly sensitive area, and because the corresponding area has dominant features, the model reduces the demand for computing resources while ensuring the accuracy of the prediction results.

[0038] In an optional implementation, historical image data is input into a pre-trained multimodal model to determine sample slices, including:

[0039] The historical image data is input into the pre-trained multimodal model to determine the characteristic heat map corresponding to the roadside perception equipment; based on the characteristic heat map, the location of the high thermal area corresponding to the specific weather type is marked to form a sample slice corresponding to the location of the high thermal area.

[0040] Specifically, a pre-trained multimodal model refers to a model that supports text and images, such as the CLIP (Contrastive Language-Image Pre-training) model. The pre-trained multimodal model outputs an image feature layer. The image feature layer passes through the fully connected layer to determine the classification of the image, and through the branched convolutional layer and the upsampling layer, obtains a feature heat map of the same size as the image input to the model and the classification corresponding to each feature heat map, that is, a feature heat map of the same size as the weather image collected by the roadside sensing equipment and the corresponding classification.

[0041] Specifically, the feature heat map is used to characterize the regional distribution of explicit weather features when the model is performing weather identification. The map information of specific weather categories in the feature heat map can be obtained by calculation, so that the area of ​​explicit weather features is used as the high heat area position in the feature heat map. Therefore, the feature corresponding to each high heat area position has a high explicit weather feature.

[0042] Specifically, the feature heat map has heat-related values, each of which is used to characterize the contribution of different regions to the final classification result. Usually, techniques such as Gard-CAM are used to emphasize the regions that have the greatest impact on classification, that is, high heat regions, through gradient information. High heat regions usually represent the feature regions that the model considers to be the most important for classification decisions. Taking Gard-CAM as an example, the steps of Gard-CAM include: forward propagation, back propagation, pooling gradients, weighted feature degrees, ReLU activation, and upsampling. Among them, in the process of forward propagation, the input image is passed through a convolutional neural network to obtain the output feature map and the final classification score. In the back propagation step, the gradient of the feature map is calculated by back propagating the classification score of a specific category. These gradients reflect the contribution of each region in the input image to the score of that category.

[0043] Specifically, based on the characteristic heat map, the position of the high heat area corresponding to the specific weather type is marked. Forming sample slices corresponding to the position of the high heat area means interpolating the characteristic heat map so that each characteristic heat map has a size consistent with the image data collected by the corresponding roadside sensing device, marking the position of the area with high dominant weather characteristics in each characteristic heat map, and using the images corresponding to these areas as sample slices, so that the area with high dominant weather characteristics is used as the basis for subsequent model training, so that the model focuses on the high heat area during the training process and ignores the non-sensitive area, thereby reducing the amount of data that the weather prediction model needs to calculate. It should be understood that the specific weather corresponding to the high heat area position refers to the prediction result generated simultaneously with the characteristic heat map when the weather image collected by the roadside sensing device is input into the pre-trained multimodal model, so that the high heat area position in the characteristic heat map represents the high dominant weather characteristics under the specific weather type.

[0044] In an optional implementation, historical image data is input into a pre-trained multimodal model to determine a correspondence between a roadside perception device and a highly sensitive area, including:

[0045] The historical image data is input into the pre-trained multimodal model to determine the characteristic heat map corresponding to the roadside perception device. Based on the characteristic heat map, the location of the high thermal area corresponding to the specific weather type is marked to form a corresponding relationship between the roadside perception device and the highly sensitive area.

[0046] Specifically, based on the characteristic heat map, the position of the high heat area corresponding to the specific weather type is marked, and the corresponding relationship between the roadside sensing device and the highly sensitive area is formed, which means that different focus areas are configured for each roadside sensing device according to the coordinate information of the position of the high heat area. For example, the focus area of ​​roadside sensing device 1 can be (x1, y1), and the focus area of ​​roadside sensing device 2 can be (x2, y1), (x3, y2), etc. It should be understood that each roadside sensing device can have one or more focus areas. Among them, since each characteristic heat map has a corresponding classification, and the position of the high heat area in each characteristic heat map is a position with a highly explicit weather feature, therefore, the classification is related to the position of the high heat area in each characteristic heat map, thereby forming a corresponding relationship between the roadside sensing device and the highly sensitive area. Still taking the example that the area of ​​interest of the roadside perception device 1 is (x1, y1) and (x2, y2), the (x1, y1) corresponding to the roadside perception device 1 can be a position with highly dominant weather characteristics relative to a cloudy day, and the (x2, y2) corresponding to the roadside perception device 1 can be a position with highly dominant weather characteristics relative to a sunny day.

[0047] Specifically, the correspondence between the roadside perception device and the highly sensitive area refers to generating configuration information in the form of json or config file based on the correspondence between the roadside perception device and the position of the highly sensitive area, so as to clarify the position of the high thermal area concerned by each roadside perception data through the configuration information.

[0048] In an optional implementation, the training process of the weather recognition model includes:

[0049] The sample slices are used as training data sets and input into the weather recognition model; after the weather recognition model focuses on the high thermal area location corresponding to the specific weather type, the training of the weather recognition model is completed to obtain a trained weather recognition model.

[0050] Specifically, the weather recognition model refers to a simple classification model, such as a ResNet model.

[0051] Specifically, using sample slices as training data sets to input the weather recognition model means using the classification corresponding to the sample slices as the slice label corresponding to each sample slice to form a weather slice sample set, thereby using the weather recognition model to train the weather slice sample set, so that when the weather recognition model converges, a trained weather recognition model is obtained. The termination condition for model training is that the model converges. Generally speaking, the loss function at this time basically no longer continues to decrease or the model reaches the maximum number of iterations. Among them, the classification corresponding to the sample slice is the classification of image prediction obtained when the historical image data is input into the pre-trained multimodal model.

[0052] In an optional implementation, based on the image data to be tested and the corresponding relationship, using the trained weather recognition model, the weather forecast result is determined, including:

[0053] Based on the corresponding relationship, the location information of the highly sensitive area corresponding to each roadside sensing device is determined; based on the location information and the image data to be tested, the weather forecast result is determined using the trained weather recognition model.

[0054] Specifically, based on the correspondence, determining the position information of the high-sensitive area corresponding to each roadside sensing device means clarifying the position of the high-sensitive area of ​​each roadside sensing device in the image data through the correspondence between the roadside sensing device and the high-sensitive area.

[0055] Specifically, based on the location information and the image data to be tested, using the trained weather recognition model, determining the weather forecast result means using the trained weather recognition model to identify the location of the high-sensitivity area of ​​each roadside sensing device in the image data to be tested, and determining the weather forecast result. For example, the high-sensitivity areas of camera 1 are a, b, c, and d. When the weather forecast result is finally determined, for example, areas a, b, and c are areas with dominant characteristics relative to sunny days, and areas b, c, and d are areas with dominant characteristics relative to cloudy days. Therefore, it is necessary to traverse areas a, b, c, and d to determine the final result.

[0056] In an optional embodiment, the method further includes:

[0057] Receive weather information and / or road information; based on the weather forecast results and the received weather information and / or road information, combined with the vehicle's trajectory information, remind the vehicle.

[0058] Specifically, receiving weather information and / or road information refers to receiving weather information and / or road information sent by, for example, a meteorological department, a road management department, or the like.

[0059] Specifically, the vehicle's trajectory information refers to the vehicle's travel trajectory information, which is predictable as the vehicle moves and the vehicle's location information is reported. Based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, reminding the vehicle means using meteorological information and / or road information such as feedback from the meteorological department and the road management department, combined with the weather forecast results and the vehicle's travel trajectory information. For example, when the weather forecast result is rainy and the meteorological information is heavy rain, and when the vehicle's trajectory information reflects that the vehicle will pass through the section during the heavy rain period, vehicle decision information such as wading reminder is generated; or when the weather forecast result is snowy and the road information is congested, and when the vehicle's trajectory information reflects that the vehicle will pass through the section during the heavy rain period, vehicle decision information such as anti-skid and increased braking distance is generated.

[0060] In an optional implementation, based on the vehicle decision information, reminding the vehicle includes:

[0061] Based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, information reminders are sent to the vehicle through roadside equipment; or, based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, subscription information is used to send information reminders to the vehicle.

[0062] Specifically, based on weather forecast results and received meteorological information and / or road information, combined with vehicle trajectory information, information reminders are provided to vehicles through roadside equipment. This means publishing information to variable information boards, roadside broadcasting and other equipment through highway-specific communication networks such as optical fibers to provide information reminders to vehicles traveling on the highway.

[0063] Specifically, based on the weather forecast results and received meteorological information and / or road information, combined with the vehicle's trajectory information, and using the subscription information, information reminders are sent to the vehicle. This means that information reminders are sent to vehicle-side users who have subscribed to smart connected application messages through 5G and other networks to achieve information reminders for the vehicle.

[0064] In an optional implementation, based on the weather forecast result and the received meteorological information and / or road information, combined with the vehicle's trajectory information, the vehicle is reminded, including:

[0065] Based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, reminder information is sent to the roadside communication module of the roadside sensing device; based on the interaction between the roadside communication module and the vehicle's on-board unit, information reminders are given to the vehicle.

[0066] Specifically, the roadside communication module that sends reminder information to the roadside sensing device based on the weather forecast results and the received meteorological information and / or road information, combined with the vehicle's trajectory information, refers to the RSU (Road Side Unit) that is sent to the roadside sensing device through a dedicated communication network such as 5G or optical fiber.

[0067] Specifically, based on the interaction between the roadside communication module and the vehicle's on-board unit, information reminders for the vehicle refer to the roadside communication module communicating with the vehicle's OBU (On board Unit) through V2X (vehicle to everything) and the RSU through DSRC (Dedicated Short Range Communication) technology to achieve information reminders for the vehicle.

[0068] By implementing this embodiment, the weather forecast results are used to determine the areas where weather changes may occur, and the received meteorological information is used to further improve the accuracy of determining the weather change areas. In combination with the received road information, the location information of special road conditions that may occur, such as congestion, collisions, and landslides, is determined, and then combined with the vehicle's trajectory area, it is determined whether the vehicle will pass through the above-mentioned area, and before the vehicle may pass through the above-mentioned area, a reminder message is sent to the vehicle, so that the owner can understand the road conditions or weather in the corresponding area in advance, thereby improving the safety of vehicle driving.

[0069] In this embodiment, a weather recognition device based on a roadside sensing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0070] This embodiment provides a weather recognition device based on a roadside sensing device, such as Figure 2 As shown, including:

[0071] The historical image data acquisition module 201 is used to acquire the historical image data collected by the roadside sensing device. The specific process can be found in the description of step S101 in the above embodiment, which will not be repeated here.

[0072] The sample slice determination module 202 is used to input the historical image data into the pre-trained multimodal model to determine the correspondence between the sample slice and the roadside perception device and the highly sensitive area. The specific process can be found in the description of step S102 in the above embodiment, which will not be repeated here.

[0073] The training module 203 is used to train the weather recognition model using the sample slices as the training data set to obtain a trained weather recognition model. The specific process can be found in the description of step S103 in the above embodiment, which will not be repeated here.

[0074] The image acquisition module 204 is used to acquire the image data to be tested collected by the roadside sensing device. The specific process can be found in the description of step S104 in the above embodiment, which will not be repeated here.

[0075] The result prediction module 205 is used to determine the weather prediction result based on the image data and the corresponding relationship using the trained weather recognition model. The specific process can be found in the description of step S105 in the above embodiment, which will not be repeated here.

[0076] The weather recognition device based on the roadside sensing equipment in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0077] In an optional embodiment, the device further comprises:

[0078] The information receiving module is used to receive weather information and / or road information. The specific process can be found in the relevant description of the above embodiment, which will not be repeated here.

[0079] The vehicle reminder module is used to remind the vehicle based on the weather forecast result and the received meteorological information and / or road information, combined with the vehicle's trajectory information. The specific process can be found in the relevant description of the above embodiment, which will not be repeated here.

[0080] The embodiment of the present invention also provides a computer device having the above Figure 2 The weather recognition device based on the roadside sensing device shown. Figure 3 , Figure 3 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 3As shown, the computer device includes: one or more processors 301, memory 302, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 3 A processor 301 is taken as an example.

[0081] The processor 301 may be a central processing unit, a network processor or a combination thereof. The processor 301 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0082] The memory 302 stores instructions executable by at least one processor 301, so that the at least one processor 301 executes the method shown in the above embodiment.

[0083] The memory 302 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 302 may optionally include a memory remotely arranged relative to the processor 301, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0084] The memory 302 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 302 may also include a combination of the above types of memory. The computer device also includes a communication interface 303, which is used for the computer device to communicate with other devices or communication networks.

[0085] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0086] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A weather recognition method based on roadside sensing equipment, characterized in that: The method comprises: Obtain historical image data collected by roadside sensing equipment; Inputting the historical image data into a pre-trained multimodal model to determine the correspondence between sample slices and the roadside perception device and the highly sensitive area; The weather recognition model is trained using the sample slices as a training data set to obtain a trained weather recognition model; Obtaining image data to be tested collected by roadside sensing equipment; Based on the image data to be tested and the corresponding relationship, using the trained weather recognition model, determining a weather forecast result; The sample slice includes: marking the location of the area with high dominant weather features in each feature heat map, and using the images corresponding to these areas as sample slices; The correspondence between the roadside sensing device and the highly sensitive area includes: determining the location of the high thermal area corresponding to the specific weather type through the marked target frame, determining the highly sensitive area and the correspondence between the roadside sensing device and the highly sensitive area.

2. The method according to claim 1, characterized in that The step of inputting the historical image data into a pre-trained multimodal model to determine sample slices includes: Inputting the historical image data into a pre-trained multimodal model to determine a characteristic heat map corresponding to the roadside perception device; Based on the characteristic heat map, the location of the high heat area corresponding to the specific weather type is marked to form a sample slice corresponding to the location of the high heat area.

3. The method according to claim 2, characterized in that The step of inputting the historical image data into a pre-trained multimodal model to determine the corresponding relationship between the roadside perception device and the highly sensitive area includes: Inputting the historical image data into a pre-trained multimodal model to determine a characteristic heat map corresponding to the roadside perception device; Based on the characteristic heat map, the location of the high thermal area corresponding to the specific weather type is marked to form a corresponding relationship between the roadside sensing equipment and the highly sensitive area.

4. The method according to claim 1, characterized in that The training process of the weather recognition model includes: Using the sample slices as a training data set, inputting into a weather recognition model; After the weather recognition model focuses on the high thermal area location corresponding to the specific weather type, the training of the weather recognition model is completed to obtain a trained weather recognition model.

5. The method according to claim 1, characterized in that The method of determining a weather forecast result based on the image data to be tested and the corresponding relationship and using the trained weather recognition model includes: Based on the corresponding relationship, determining the location information of the highly sensitive area corresponding to each of the roadside sensing devices; Based on the location information and the image data to be tested, a weather forecast result is determined using a trained weather recognition model.

6. The method according to claim 1, characterized in that The method further comprises: receiving weather information and / or road information; Based on the weather forecast result and the received meteorological information and / or the road information, combined with the vehicle's trajectory information, the vehicle is reminded.

7. The method according to claim 6, characterized in that The reminding the vehicle based on the weather forecast result and the received meteorological information and / or the road information in combination with the vehicle's trajectory information includes: Based on the weather forecast result and the received meteorological information and / or the road information, combined with the vehicle's trajectory information, sending a reminder message to a roadside communication module of a roadside sensing device; Based on the interaction between the roadside communication module and the vehicle's on-board unit, information reminders are provided to the vehicle.

8. A weather recognition device based on roadside sensing equipment, characterized in that: The device comprises: A historical image data acquisition module, used to acquire historical image data collected by roadside sensing equipment; A sample slice determination module is used to input the historical image data into a pre-trained multimodal model to determine sample slices and the correspondence between the roadside perception device and the highly sensitive area; wherein the sample slices include: marking the location of the area with high dominant weather features in each feature heat map, and using the images corresponding to these areas as sample slices; the correspondence between the roadside perception device and the highly sensitive area includes: determining the location of the high thermal area corresponding to a specific weather type through a marked target box, determining the highly sensitive area and the correspondence between the roadside perception device and the highly sensitive area; A training module, used to train a weather recognition model using the sample slices as a training data set to obtain a trained weather recognition model; The test image acquisition module is used to acquire the test image data collected by the roadside perception device; The result prediction module is used to determine the weather prediction result based on the image data to be tested and the corresponding relationship, using the trained weather recognition model.

9. The device according to claim 8, characterized in that The device further comprises: An information receiving module, used for receiving weather information and / or road information; The vehicle reminder module is used to remind the vehicle based on the weather forecast result and the received meteorological information and / or the road information, combined with the vehicle's trajectory information.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the weather recognition method based on a roadside perception device according to any one of claims 1 to 7 by executing the computer instructions.

Images