A roadside parking scene analysis method and system for night scenes

Abstract

The present invention discloses a roadside parking scene analysis method and system for night scenes, and relates to the field of intelligent parking management. The method comprises: firstly, constructing a connection between two different modal data, image and event, by utilizing an image motion feature extraction model, and then utilizing a content feature extraction model to reduce the difference between data in two different domains, daytime scenes and nighttime scenes, taking into account various requirements for model efficiency, robustness, accuracy, etc. in roadside scenes, and at the same time adopting a semantic segmentation model structure to realize the scene analysis task of night scenes, realize the category and mask output of different targets in roadside scenes, improve the semantic segmentation accuracy of night scenes, and contribute to the traffic management of roadside parking scenes.

Description

Technical Field

[0001] The present invention relates to the field of intelligent parking management, and in particular to a roadside parking scene analysis method and system for nighttime scenes. Background Art

[0002] In recent years, with the development of social economy, people's demand for convenience in daily travel has continued to increase, and the number of cars has continued to increase. However, the increase in the number of parking spaces is far less than the increase in the number of vehicles, resulting in parking difficulties and parking chaos, which has become one of the main problems that need to be solved in the development of urban transportation in my country. Among them, setting up parking spaces on the roadside, using intelligent sensing devices to collect data such as video images, and processing and analyzing the data through computer vision algorithms to achieve intelligent parking management is currently a mainstream solution. At present, it is mainly achieved by installing high-position video cameras on the roadside for data collection, and then using visual algorithms such as semantic segmentation algorithms to realize the analysis of roadside parking scenes.

[0003] However, compared with other spectra, the visible light imaging range is significantly narrower and is only effective in environments with good lighting and high visibility. It may fail in special environments such as night. In low-light scenes such as night, the image quality captured by the camera is low, the visibility of vehicle targets is low, and there is too much noise information in the image, resulting in low judgment accuracy for tasks such as semantic segmentation in low-light scenes. For the enhancement of such low-light images, currently, image enhancement is mainly achieved based on visible light images generated by the camera by utilizing algorithms such as generative adversarial networks or diffusion models. However, in many cases with poor lighting conditions, low-light images cannot be restored, and some noise information is easily introduced, resulting in low accuracy in parsing roadside parking scenes at night. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method and system for analyzing roadside parking scenes in night scenes, which can solve the problem of low accuracy of existing roadside parking scene analysis in night scenes.

[0005] To achieve the above object, the present invention provides a roadside parking scene analysis method for night scenes, the method comprising:

[0006] Two adjacent frames of images in the daytime scene are processed through the image motion feature extraction model to generate relative motion data of the daytime scene;

[0007] The image content feature extraction model is used to extract image content features from each image frame based on the differences in image style features during the day and at night.

[0008] The relative motion data and image content features of the daytime scene are used as inputs of a preset semantic segmentation model to generate roadside parking scene analysis data for a nighttime scene.

[0009] Furthermore, the step of processing two adjacent frames of images in a daytime scene by using an image motion feature extraction model to generate relative motion data of the daytime scene includes:

[0010] According to the formula Calculate, where Represents the relative motion data in the daytime scene, and the two adjacent frames of images after grayscale processing are expressed as , , , , , Indicates calculating two adjacent frames of images in the log domain The difference, is a constant quantity, It represents the result of truncating the extremely large and extremely small data using two hyperparameters. Indicates the normalization result;

[0011] According to the formula The relative motion data is denoised, where is the relative motion data of the daytime scene after denoising, Represents the style transfer model.

[0012] Furthermore, the step of extracting image content features from each image frame by using the image content feature extraction model according to the difference in image style features during the day and at night includes:

[0013] According to the formula Calculate, where is the image content feature, I represents the grayscale image, is the number of pixels that the grayscale image coordinates move, The coordinates of the grayscale image I are randomly shifted left or right The result of pixels, The coordinates of the grayscale image I are randomly moved up or down The result of pixels, () indicates the result of image shift.

[0014] Furthermore, the step of using the relative motion data and image content features of the daytime scene as input of a preset semantic segmentation model to generate roadside parking scene analysis data for a nighttime scene includes: extracting image features from the input image through an image feature encoder in the preset semantic segmentation model; extracting event features and / or image content features from the relative motion data and / or image content features of the daytime scene through an event feature encoder in the preset semantic segmentation model; fusing the extracted image features, and / or image content features, and / or event features through a cross-modal feature fuser in the preset semantic segmentation model, and outputting a prediction result.

[0015] Furthermore, before the step of processing two adjacent frames of images in a daytime scene by using an image motion feature extraction model to generate relative motion data of the daytime scene, the method further includes:

[0016] According to the formula The image motion feature extraction model, image content feature extraction model, and preset semantic segmentation model are trained and updated, among which, , , , are weight coefficients, represents the semantic segmentation prediction result of the input image I, represents the semantic segmentation prediction result of the input event E, Represents image content The semantic segmentation prediction results are: Represents the semantic segmentation prediction result of the fused image features and event features, Represents the semantic segmentation prediction result of the fused image features and image content features, cross entropy loss function The calculation formula is , C represents the number of target segmentation categories, It indicates the probability that the predicted sample belongs to class c. Represents the true label category distribution.

[0017] Furthermore, the present invention provides a roadside parking scene analysis system for nighttime scenes, the system comprising:

[0018] A generation module, used to process two adjacent frames of images in a daytime scene through an image motion feature extraction model to generate relative motion data of the daytime scene;

[0019] An extraction module, used for extracting image content features from each image frame according to the difference in image style features between daytime and nighttime, through an image content feature extraction model;

[0020] The generation module is also used to use the relative motion data and image content features of the daytime scene as inputs of a preset semantic segmentation model to generate roadside parking scene analysis data for a nighttime scene.

[0021] Furthermore, the generating module is specifically used to generate Calculate, where Represents the relative motion data in the daytime scene, and the two adjacent frames of images after grayscale processing are expressed as , , , , , Indicates calculating two adjacent frames of images in the log domain The difference, is a constant quantity, It represents the result of truncating the extremely large and extremely small data using two hyperparameters. Indicates the normalization result;

[0022] According to the formula The relative motion data is denoised, where is the relative motion data of the daytime scene after denoising, Represents the style transfer model.

[0023] Furthermore, the extraction module is specifically used to extract Calculate, where is the image content feature, I represents the grayscale image, is the number of pixels that the grayscale image coordinates move, The coordinates of the grayscale image I are randomly shifted left or right The result of pixels, The coordinates of the grayscale image I are randomly moved up or down The result of pixels, () indicates the result of image shift.

[0024] Furthermore, the generation module is specifically used to extract image features from the input image through the image feature encoder in the preset semantic segmentation model; extract event features and / or image content features from the relative motion data and / or image content features of the daytime scene through the event feature encoder in the preset semantic segmentation model; fuse the extracted image features, and / or image content features, and / or event features through the cross-modal feature fuser in the preset semantic segmentation model, and output a prediction result.

[0025] Furthermore, the system also includes: a training module;

[0026] The training module is used to The image motion feature extraction model, image content feature extraction model, and preset semantic segmentation model are trained and updated, among which, , , , are weight coefficients, represents the semantic segmentation prediction result of the input image I, represents the semantic segmentation prediction result of the input event E, Represents image content The semantic segmentation prediction results are: Represents the semantic segmentation prediction result of the fused image features and event features, Represents the semantic segmentation prediction result of the fused image features and image content features, cross entropy loss function The calculation formula is , C represents the number of target segmentation categories, It indicates the probability that the predicted sample belongs to class c. Represents the true label category distribution.

[0027] The present invention provides a method and system for analyzing roadside parking scenes at night. First, an image motion feature extraction model is used to construct a connection between two different modal data, image and event. Then, a content feature extraction model is used to reduce the difference between data in two different domains, daytime scenes and nighttime scenes. The requirements for model efficiency, robustness, accuracy and other aspects in roadside scenes are taken into account. At the same time, a semantic segmentation model structure is used to implement the scene analysis task of nighttime scenes, and the category and mask output of different targets in roadside scenes are realized, thereby improving the semantic segmentation accuracy of nighttime scenes and facilitating traffic management of roadside parking scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of a roadside parking scene analysis method for nighttime scenes provided by the present invention;

[0029] Figure 2 It is a schematic diagram of a roadside parking scene analysis system for nighttime scenes provided by the present invention. DETAILED DESCRIPTION

[0030] The device structure and implementation of the present invention are further described in detail below through the accompanying drawings and embodiments.

[0031] The present invention provides a method for analyzing roadside parking scenes at night. Figure 1 As shown, the specific steps include:

[0032] 101. Two adjacent frames of images in the daytime scene are processed through the image motion feature extraction model to generate relative motion data of the daytime scene.

[0033] For the embodiment of the present invention, step 101 may specifically include: according to the formula Calculate, where Represents the relative motion data in the daytime scene, and the two adjacent frames of images after grayscale processing are expressed as , , , , , Indicates calculating two adjacent frames of images in the log domain The difference, is a constant quantity, It represents the result of truncating the extremely large and extremely small data using two hyperparameters. Indicates the normalization result;

[0034] According to the formula The relative motion data is denoised, where is the relative motion data of the daytime scene after denoising, Represents the style transfer model.

[0035] It should be noted that due to the lack of event data in daytime scenes, but considering that the event data is generated due to the relative motion of the camera and the scene, an image motion feature extraction module is constructed to generate the relative motion data of the daytime scenes; further, the relative motion data of the daytime scenes are obtained through the above steps. In addition, considering that there is noise in the event camera data in the nighttime scenes, in order to simulate more realistic relative motion data, the style of the real event camera data is added to the relative motion data by utilizing the style transfer model.

[0036] For the embodiment of the present invention, before step 101, the following may be further included: constructing a paired image-event dataset in a dark roadside parking scene for model training; specifically, the dataset may be acquired in two ways, one is the acquisition of real data, and the other is by using open source data or simulation data; for the acquisition of real data of night scenes, an event camera and an ordinary camera are used, and the camera is fixed on a stable tripod to ensure that the captured images are aligned at the pixel level, and then the roadside parking scene is photographed to obtain an image and event sequence pair in the night scene; and a daytime scene image under the same viewing angle;

[0037] For open source data, including but not limited to EventScape, DELIVER, DDD17, DSEC and other event camera segmentation data sets; EventScape and DELIVER are simulation data sets, and semantic segmentation annotations are obtained by using simulators or larger model prediction results; wherein, the roadside parking scenes include major urban roads, intersections, roadside parking areas, key areas such as schools, hospitals and other places of traffic locations such as parking lots at the entrance; the data annotation is semantically segmented, and polygon tools are used for data annotation. The annotation categories include but are not limited to pedestrians, vehicles, non-motor vehicles, lane lines, traffic signs, green plants and other categories of target annotations, and the vehicles specifically include cars, buses, taxis and other categories.

[0038] 102. The image content feature extraction model extracts image content features from each image frame based on the differences in image style features between daytime and nighttime.

[0039] For the embodiment of the present invention, step 102 may specifically include: according to the formula Calculate, where is the image content feature, I represents the grayscale image, is the number of pixels that the grayscale image coordinates move, The coordinates of the grayscale image I are randomly shifted left or right The result of pixels, The coordinates of the grayscale image I are randomly moved up or down The result of pixels, () indicates the result of image shift.

[0040] It should be noted that the function of the image content feature extractor is to eliminate the style features of daytime and nighttime to obtain the content features of the image.

[0041] 103. Use the relative motion data of the daytime scene and the image content features as inputs of a preset semantic segmentation model to generate roadside parking scene analysis data for a nighttime scene.

[0042] For the embodiment of the present invention, step 103 may specifically include: extracting image features from the input image through an image feature encoder in a preset semantic segmentation model; extracting event features and / or image content features from the relative motion data and / or image content features of the daytime scene through an event feature encoder in a preset semantic segmentation model; fusing the extracted image features, and / or image content features, and / or event features through a cross-modal feature fuser in a preset semantic segmentation model, and outputting a prediction result.

[0043] Specifically, the embodiment of the present invention adopts a semantic segmentation model based on DAFormer, which adopts a model training method based on self-supervised training. The model structure includes a student network and a network of teachers ;

[0044] Among them, the student network is used to predict the semantic segmentation prediction results of the input data, and the teacher network is used to provide pseudo labels for the target domain, namely the night scene input data; both the student network and the teacher network are composed of two feature encoders, including an image feature encoder and an event feature encoder, a cross-modal feature fusion device, and a feature decoder; the input consists of three parts, I represents the input image, E represents the input event, Represents the content of the input image, where the image feature encoder extracts features from the input image I; the event feature encoder extracts features from the input event E or image content The cross-modal feature fuser fuses the extracted image features and event features, or the extracted image features and image content features. Finally, the decoder predicts the unfused features and fused features, including ,in, Represents the semantic segmentation prediction result of the input image I; Represents the semantic segmentation prediction result of the input event E; Represents image content Semantic segmentation prediction results; Represents the semantic segmentation prediction result of the fused image features and event features; The semantic segmentation prediction result of the fused image features and image content features represents the semantic segmentation prediction result of the input image I;

[0045] For the event feature encoder, randomly select from the input event E or image content The image feature encoder and event feature encoder use a hierarchical transformer-based encoder module to generate high-resolution coarse features and low-resolution fine features. The feature fusion module uses multiple attention modules to fuse data of different modalities. The feature decoder uses a lightweight multi-layer perceptron structure to generate the final semantic segmentation mask.

[0046] For the embodiment of the present invention, based on the semantic segmentation model of the roadside parking scene of the above-mentioned night scene, the night scene data is input, and the semantic segmentation task prediction is performed to realize the semantic segmentation of different categories of targets in the night scene; the night scene data includes night scene images and event data, and the semantic segmentation represents the pixel-by-pixel classification of the targets in the image to obtain the mask information and category information of the targets; the categories mainly include green plants, berth lines, lane lines, manhole covers, zebra crossings, iron grates and other categories. By utilizing the semantic segmentation algorithm, the positions of targets such as green plants and berth lines can be obtained more accurately, thereby realizing the roadside scene parsing task.

[0047] It should be noted that the image motion feature extraction model, image content feature extraction model, and preset semantic segmentation model used in the embodiment of the present invention are pre-trained. The image motion feature extraction model, image content feature extraction model, and preset semantic segmentation model are trained and updated, among which, , , , are weight coefficients, represents the semantic segmentation prediction result of the input image I, represents the semantic segmentation prediction result of the input event E, Represents image content The semantic segmentation prediction results are: Represents the semantic segmentation prediction result of the fused image features and event features, Represents the semantic segmentation prediction result of the fused image features and image content features, cross entropy loss function The calculation formula is , C represents the number of target segmentation categories, It indicates the probability that the predicted sample belongs to class c. Represents the true label category distribution.

[0048] An embodiment of the present invention provides a method for analyzing roadside parking scenes at night. First, an image motion feature extraction model is used to construct a connection between two different modal data, image and event. Then, a content feature extraction model is used to reduce the difference between data in two different domains, daytime scenes and nighttime scenes. This method takes into account various requirements for model efficiency, robustness, accuracy, etc. in roadside scenes. At the same time, a semantic segmentation model structure is used to implement scene analysis tasks for nighttime scenes, and the categories and masks of different targets in roadside scenes are output, thereby improving the semantic segmentation accuracy of nighttime scenes and facilitating traffic management in roadside parking scenes.

[0049] As Figure 1The specific implementation of the method shown in the figure, the embodiment of the present invention provides a roadside parking scene analysis system for night scenes, such as Figure 2 As shown, the system includes: a generating module 21, which is used to process two adjacent frames of images in a daytime scene through an image motion feature extraction model to generate relative motion data of the daytime scene;

[0050] An extraction module 22, configured to extract image content features from each image frame according to the difference in image style features between daytime and nighttime through an image content feature extraction model;

[0051] The generation module 21 is further used to use the relative motion data and image content features of the daytime scene as inputs of a preset semantic segmentation model to generate roadside parking scene analysis data for a nighttime scene.

[0052] Furthermore, the generating module 21 is specifically used to generate the Calculate, where Represents the relative motion data in the daytime scene, and the two adjacent frames of images after grayscale processing are expressed as , , , , , Indicates calculating two adjacent frames of images in the log domain The difference, is a constant quantity, It represents the result of truncating the extremely large and extremely small data using two hyperparameters. Indicates the normalization result;

[0053] According to the formula The relative motion data is denoised, where is the relative motion data of the daytime scene after denoising, Represents the style transfer model.

[0054] Furthermore, the extraction module 22 is specifically configured to extract Calculate, where is the image content feature, I represents the grayscale image, is the number of pixels that the grayscale image coordinates move, The coordinates of the grayscale image I are randomly shifted left or right The result of pixels, The coordinates of the grayscale image I are randomly moved up or down The result of pixels, () indicates the result of image shift.

[0055] Furthermore, the generation module 21 is specifically used to extract image features from the input image through an image feature encoder in a preset semantic segmentation model; extract event features and / or image content features from the relative motion data and / or image content features of the daytime scene through an event feature encoder in a preset semantic segmentation model; fuse the extracted image features, and / or image content features, and / or event features through a cross-modal feature fuser in a preset semantic segmentation model, and output a prediction result.

[0056] Furthermore, the system further comprises: a training module 23;

[0057] The training module 23 is used to The image motion feature extraction model, image content feature extraction model, and preset semantic segmentation model are trained and updated, among which, , , , are weight coefficients, represents the semantic segmentation prediction result of the input image I, represents the semantic segmentation prediction result of the input event E, Represents image content The semantic segmentation prediction results are: Represents the semantic segmentation prediction result of the fused image features and event features, Represents the semantic segmentation prediction result of the fused image features and image content features, cross entropy loss function The calculation formula is , C represents the number of target segmentation categories, It indicates the probability that the predicted sample belongs to class c. Represents the true label category distribution.

[0058] An embodiment of the present invention provides a roadside parking scene analysis system for night scenes. First, an image motion feature extraction model is used to construct a connection between two different modal data, image and event. Then, a content feature extraction model is used to reduce the difference between data in two different domains, daytime scenes and nighttime scenes. This takes into account various requirements for model efficiency, robustness, accuracy, etc. in roadside scenes. At the same time, a semantic segmentation model structure is used to implement scene analysis tasks for night scenes, and the categories and masks of different targets in roadside scenes are output, thereby improving the semantic segmentation accuracy of night scenes and facilitating traffic management in roadside parking scenes.

[0059] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0060] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0061] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0062] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0063] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps mentioned above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0064] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0065] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0066] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and Blu-ray disk. Disks usually copy data magnetically, while discs usually copy data optically with lasers. Combinations of the above may also be included in computer-readable media.

[0067] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for analyzing roadside parking scenes at night, characterized in that: The method comprises: Two adjacent frames of images in the daytime scene are processed through the image motion feature extraction model to generate relative motion data of the daytime scene; The step of processing two adjacent frames of images in a daytime scene by using an image motion feature extraction model to generate relative motion data of the daytime scene includes: According to formula E ME =F filter (I k-1 , I k ) is calculated, where E ME Represents the relative motion data in the daytime scene, and the two adjacent frames of images after grayscale processing are represented as I k-1 and I k , F filter (I k-1 , I k )=F norm (F clipIgn (F LogDiff (I k-1 , I k ))), F LogDiff (I k-1 , I k )=ln(I k-1 +∈)-ln(I k +∈), F clipIgn (x)=min(|x|,α)·sgn(x)·l(|x|>β), F LogDiff (I k-1 , I k ) represents the calculation of two adjacent frames of image I in the log domain k-1 with I k The difference of ∈ is a constant, F clipIgn (x) represents the result of truncating the maximum and minimum data using two hyperparameters, F norm (x) represents the normalization result, x is the parameter variable, α and β are constants; According to the formula The relative motion data is denoised, where is the relative motion data of the daytime scene after denoising, Represents the style transfer model; The image content feature extraction model is used to extract image content features from each image frame based on the differences in image style features during the day and at night. Using the relative motion data of the daytime scene and the image content features as inputs of a preset semantic segmentation model to generate roadside parking scene analysis data of a nighttime scene; The step of using the relative motion data and image content features of the daytime scene as inputs of a preset semantic segmentation model to generate nighttime roadside parking scene analysis data includes: Extract image features from the input image through the image feature encoder in the preset semantic segmentation model; Extracting event features and / or image content features from the relative motion data and / or image content features of the daytime scene through an event feature encoder in a preset semantic segmentation model; The extracted image features, and / or image content features, and / or event features are fused through a cross-modal feature fuser in a preset semantic segmentation model to output a prediction result.

2. The method for analyzing roadside parking scenes at night according to claim 1, characterized in that: The step of extracting image content features from each image frame according to the difference in image style features between daytime and nighttime by using the image content feature extraction model comprises: According to the formula Calculate, where I CE is the image content feature, I represents the grayscale image, γ is the number of pixels moved by the grayscale image coordinates, x represents the x-axis coordinate, y represents the y-axis coordinate, and I x±γ is the result of randomly shifting the coordinates of the grayscale image I to the left or right by γ pixels, I y±γ is the result of randomly moving the coordinates of the grayscale image I up or down by γ pixels, F filter () indicates the result of image shift.

3. The method for analyzing roadside parking scenes at night according to claim 1, characterized in that: Before the step of processing two adjacent frames of images in a daytime scene by using an image motion feature extraction model to generate relative motion data of the daytime scene, the method further includes: According to the formula The image motion feature extraction model, image content feature extraction model, and preset semantic segmentation model are trained and updated, where α I , α E , α fusion are weight coefficients, P I represents the semantic segmentation prediction result of the input image I, P E represents the semantic segmentation prediction result of the input event E, Indicates the image content I CE The semantic segmentation prediction result, P I+E Represents the semantic segmentation prediction result of the fused image features and event features, and the cross entropy loss function L seg The calculation formula is C represents the number of target segmentation categories, p c Indicates the probability that the predicted sample belongs to class c, y c Represents the true label category distribution.

4. A roadside parking scene analysis system for night scenes, characterized in that: The system comprises: A generation module, used to process two adjacent frames of images in a daytime scene through an image motion feature extraction model to generate relative motion data of the daytime scene; The generating module is specifically used to generate the ME =F filter (I k-1 , I k ) is calculated, where E ME Represents the relative motion data in the daytime scene, and the two adjacent frames of images after grayscale processing are represented as I k-1 and I k , F filter (I k-1 , I k )=F norm (F clipIgn (F LogDiff (I k-1 , I k ))), F LogDiff (I k-1 , I k )=ln(I k-1 +∈)-ln(I k +∈), F clipIgn (x)=min(|x|,α)·sgn(x)·l(|x|>β), F LogDiff (I k-1 , I k ) represents the calculation of two adjacent frames of image I in the log domain k-1 with I k The difference of ∈ is a constant, F clipIgn (x) represents the result of truncating the maximum and minimum data using two hyperparameters, F norm (x) represents the normalization result, x is the parameter, α and β are constants; according to the formula The relative motion data is denoised, where is the relative motion data of the daytime scene after denoising, Represents the style transfer model; An extraction module, used for extracting image content features from each image frame according to the difference in image style features between daytime and nighttime, through an image content feature extraction model; The generation module is further used to use the relative motion data and image content features of the daytime scene as inputs of a preset semantic segmentation model to generate roadside parking scene analysis data of a nighttime scene; The generation module is specifically used to extract image features from the input image through the image feature encoder in the preset semantic segmentation model; extract event features and / or image content features from the relative motion data and / or image content features of the daytime scene through the event feature encoder in the preset semantic segmentation model; fuse the extracted image features, and / or image content features, and / or event features through the cross-modal feature fuser in the preset semantic segmentation model, and output a prediction result.

5. The roadside parking scene analysis system for nighttime scenes according to claim 4, characterized in that: The extraction module is specifically used to extract Calculate, where I CE is the image content feature, I represents the grayscale image, γ is the number of pixels moved by the grayscale image coordinates, x represents the x-axis coordinate, y represents the y-axis coordinate, and I x±γ is the result of randomly shifting the coordinates of the grayscale image I to the left or right by γ pixels, I y±γ is the result of randomly moving the coordinates of the grayscale image I up or down by γ pixels, F filter () indicates the result of image shift.

6. The roadside parking scene analysis system for nighttime scenes according to claim 4, characterized in that: The system also includes: a training module; The training module is used to The image motion feature extraction model, image content feature extraction model, and preset semantic segmentation model are trained and updated, where α I , α E , α fusion are weight coefficients, P I represents the semantic segmentation prediction result of the input image I, P E represents the semantic segmentation prediction result of the input event E, Indicates the image content I CE The semantic segmentation prediction result, P I+E Represents the semantic segmentation prediction result of the fused image features and event features, and the cross entropy loss function L seg The calculation formula is C represents the number of target segmentation categories, p c Indicates the probability that the predicted sample belongs to class c, y c Represents the true label category distribution.

Images