Parking scene change detection method and system

Abstract

The present invention discloses a parking scene change detection method and system. The method comprises: sequentially passing a bi-temporal scene image through a first-layer network structure, a second-layer network structure, and a third-layer network structure of a scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map; upsampling the feature difference map and performing a jump connection with the third feature map to obtain a first jump connection feature; upsampling the first jump connection feature and performing a jump connection with the second feature map to obtain a second jump connection feature; upsampling the second jump connection feature and performing a jump connection with the first feature map to obtain a third jump connection feature; pixel-by-pixel classification of the third jump connection feature to generate a bi-temporal scene change region image; optimizing parameters of the scene change detection model based on the bi-temporal scene change region image and a change label image; and performing parking scene change detection on multiple sets of bi-temporal scene images to be tested to obtain corresponding change region images.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a parking scene change detection method and system. Background Art

[0002] With the development of my country's economy and technology, the number of motor vehicles in the country is rapidly increasing. This growth is compounded by the challenges of urban development, resource limitations, the scale of parking facilities, and the uneven utilization of parking resources across time and space. This has led to increasingly prominent parking problems, including chaotic and difficult parking. Given the current imbalance between supply and demand and the shortage of parking spaces, optimizing and improving parking management is imperative.

[0003] In parking management, the application of deep learning technology has actively promoted the development of smart urban transportation. In video-based parking management, accurate understanding of parking scene status is a prerequisite for intelligent management. Therefore, close attention must be paid to the changing state of parking scenes. The changing state of parking scenes consists of two or more parking scenes at the same location. Identifying which areas within a scene have changed is key to determining the changing state of parking scenes at that location.

[0004] In image processing, detecting and identifying changes in pairs of images of the same scene at different times is known as change detection. Traditional methods use direct object comparison change detection, directly comparing multiple pixels with spatial and texture information, directly comparing the differences between pixels at the same geographic location but at different times. However, due to external factors such as light intensity, background fluctuations, and shadows, image changes are diverse and ambiguous. Traditional direct object comparison change detection methods are unable to accurately detect changes in parking scenes. Summary of the Invention

[0005] The purpose of the present invention is to solve the technical problem that the traditional direct object comparison change detection method cannot accurately detect the changes in the parking scene. To achieve the above purpose, the present invention provides a parking scene change detection method and system.

[0006] The present invention provides a parking scene change detection method, comprising:

[0007] Constructing a parking scene dataset, the parking scene dataset including multiple groups of scene images, each group of scene images including a bi-temporal scene image and a change label image;

[0008] Passing the dual-phase scene image sequentially through the first layer network structure, the second layer network structure, and the third layer network structure of the scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map;

[0009] Upsampling the feature difference map and performing a jump connection with the third feature map to obtain a first jump connection feature, upsampling the first jump connection feature and performing a jump connection with the second feature map to obtain a second jump connection feature, and upsampling the second jump connection feature and performing a jump connection with the first feature map to obtain a third jump connection feature;

[0010] performing pixel-by-pixel classification on the third skip connection feature to generate a bi-temporal scene change region image, and optimizing parameters of the scene change detection model based on the bi-temporal scene change region image and the change label image to obtain a trained scene change detection model;

[0011] Parking scene change detection is performed on multiple groups of dual-phase scene images to be tested according to the trained scene change detection model to obtain change area images corresponding to the dual-phase scene images to be tested.

[0012] In one embodiment, the step of sequentially passing the dual-temporal scene image through a first-layer network structure, a second-layer network structure, and a third-layer network structure of a scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map includes:

[0013] Inputting the multi-temporal scene image into the first-layer network structure, outputting a first low-level feature map and a second low-level feature map, and performing feature fusion on the first low-level feature map and the second low-level feature map according to the attention mechanism method to obtain the first feature map;

[0014] Inputting the first low-level feature map and the second low-level feature map into the second-layer network structure, outputting a first intermediate feature map and a second intermediate feature map, and performing feature fusion on the first intermediate feature map and the second intermediate feature map according to the attention mechanism method to obtain the second feature map;

[0015] Inputting the first intermediate feature map and the second intermediate feature map into the third-layer network structure, outputting a first high-level feature map and a second high-level feature map, and performing feature fusion on the first high-level feature map and the second high-level feature map according to the attention mechanism method to obtain the third feature map;

[0016] Calculate the feature difference between the first high-level feature map and the second high-level feature map to obtain the feature difference map.

[0017] In one embodiment, upsampling the feature difference map and performing a jump connection with the third feature map to obtain a first jump connection feature, upsampling the first jump connection feature and performing a jump connection with the second feature map to obtain a second jump connection feature, and upsampling the second jump connection feature and performing a jump connection with the first feature map to obtain a third jump connection feature, includes:

[0018] Performing upsampling processing on the feature difference map to obtain an upsampled difference map;

[0019] Inputting the upsampled difference map and the third feature map into a skip connection convolutional network to form the first skip connection feature;

[0020] Upsampling the first skip connection feature and inputting it and the second feature map into the skip connection convolutional network to form the second skip connection feature;

[0021] The second skip connection feature is upsampled and inputted into the skip connection convolutional network together with the first feature map to form the third skip connection feature.

[0022] In one embodiment, after performing parking scene change detection on a plurality of sets of dual-temporal scene images to be tested based on the trained scene change detection model and obtaining the changed area images corresponding to the dual-temporal scene images to be tested, the method further includes:

[0023] Mapping the change region image pixel by pixel to the parking scene image corresponding to the last moment in the dual-temporal scene image to be measured, to generate a change mask region corresponding to the parking scene image;

[0024] According to the change mask area and the berth area in the parking scene image, a change direction and a change amount of the change mask area relative to the berth area are obtained.

[0025] In one embodiment, the parking scene dataset is constructed, wherein the parking scene dataset includes multiple groups of scene images, each group of scene images includes a bi-temporal scene image and a change label image, including:

[0026] Acquire the multi-temporal scene images of a plurality of different regions, and reconstruct a plurality of the dual-temporal scene images of each region based on the multi-temporal scene images of each region, mark the change position of each dual-temporal scene image, and obtain the change region;

[0027] Converting the changed region into a binary label image to obtain the changed label image;

[0028] The plurality of dual-phase scene images and the corresponding change label images are converted into images of the same size to construct the parking scene dataset.

[0029] The present invention provides a parking scene change detection system, comprising:

[0030] A data construction module is used to construct a parking scene dataset, wherein the parking scene dataset includes multiple groups of scene images, each group of scene images includes a dual-phase scene image and a change label image;

[0031] a feature extraction module, configured to sequentially pass the dual-temporal scene image through a first-layer network structure, a second-layer network structure, and a third-layer network structure of a scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map;

[0032] a feature skip connection module, configured to perform a skip connection on the feature difference map after upsampling and the third feature map to obtain a first skip connection feature, perform a skip connection on the first skip connection feature after upsampling and the second feature map to obtain a second skip connection feature, and perform a skip connection on the second skip connection feature after upsampling and the first feature map to obtain a third skip connection feature;

[0033] a change detection model forming module, configured to perform pixel-by-pixel classification on the third skip connection feature to generate a bi-temporal scene change region image, and optimize parameters of the scene change detection model based on the bi-temporal scene change region image and the change label image to obtain a trained scene change detection model;

[0034] The change detection module is used to perform parking scene change detection on multiple groups of dual-phase scene images to be tested based on the trained scene change detection model, and obtain change area images corresponding to the dual-phase scene images to be tested.

[0035] In one embodiment, the feature extraction module includes:

[0036] A first feature acquisition module is configured to input the bi-temporal scene image into the first-layer network structure, output a first low-level feature map and a second low-level feature map, and perform feature fusion on the first low-level feature map and the second low-level feature map according to an attention mechanism method to obtain the first feature map;

[0037] A second feature acquisition module is configured to input the first low-level feature map and the second low-level feature map into the second layer network structure, output a first intermediate feature map and a second intermediate feature map, and perform feature fusion on the first intermediate feature map and the second intermediate feature map according to the attention mechanism method to obtain the second feature map;

[0038] A third feature acquisition module is configured to input the first intermediate feature map and the second intermediate feature map into the third layer network structure, output a first high-level feature map and a second high-level feature map, and perform feature fusion on the first high-level feature map and the second high-level feature map according to the attention mechanism method to obtain the third feature map;

[0039] The feature difference acquisition module is used to calculate the feature difference between the first high-level feature map and the second high-level feature map to obtain the feature difference map.

[0040] In one embodiment, the feature skip connection module includes:

[0041] an upsampling module, configured to perform upsampling processing on the feature difference map to obtain an upsampled difference map;

[0042] a first skip connection feature acquisition module, configured to input the upsampled difference map and the third feature map into a skip connection convolutional network to form the first skip connection feature;

[0043] a second skip connection feature acquisition module, configured to upsample the first skip connection feature and input the first skip connection feature and the second feature map into the skip connection convolutional network to form the second skip connection feature;

[0044] The third skip connection feature acquisition module is used to input the second skip connection feature and the first feature map into the skip connection convolutional network after upsampling to form the third skip connection feature.

[0045] In one embodiment, the system further comprises:

[0046] a change mask region generating module, configured to map the change region image pixel by pixel to the parking scene image corresponding to the last moment in the dual-temporal scene image to be measured, and generate a change mask region corresponding to the parking scene image;

[0047] The change acquisition module is used to obtain the change direction and change amount of the change mask area relative to the berth area based on the change mask area and the berth area in the parking scene image.

[0048] In one embodiment, the data construction module includes:

[0049] a marking module, configured to obtain the multi-temporal scene images of a plurality of different regions, and reconstruct a plurality of the dual-temporal scene images of each region based on the multi-temporal scene images of each region, mark a change position of each of the dual-temporal scene images, and obtain a change region;

[0050] a change label image acquisition module, configured to convert the change region into a binary label image to obtain the change label image;

[0051] The size change module is used to convert the plurality of the dual-phase scene images and the corresponding change label images into images of the same size to construct the parking scene dataset.

[0052] In the above-mentioned parking scene change detection method and system, by constructing a parking scene dataset, marking the change areas of interest, ignoring the changes that are not of interest, and reducing the interference of pseudo-changes in the parking scene. The scene change detection model is used to learn the features of the dual-phase scene image. After the three-layer network structure of low, medium and high, the feature map of each layer and the feature difference map of the last layer are extracted, and step-by-step jump connection is implemented between the feature difference map and multiple feature maps to obtain the third jump connection feature formed after the jump connection. The third jump connection feature is classified pixel by pixel to generate the dual-phase scene change area image predicted by the scene change detection model, and compared with the change label image annotated in the dataset to realize the training of the scene change detection model, which is used for the subsequent detection of the dual-phase scene image to be tested, and the corresponding change area image is obtained, thereby realizing the detection of the change area of ​​the parking scene.

[0053] Through the parking scene change detection method provided by the present invention, the dual-phase scene image undergoes feature extraction at multiple different network levels, fully learns the features of each detail in the image, and jumps the features extracted from each layer of the network structure to fully integrate the various feature details in the image, and performs pixel-by-pixel classification to obtain the changed area image, reducing the influence of external factors such as light intensity, background fluctuations, and shadows. It can better solve the problem of low change detection accuracy caused by diverse and unclear image changes, and can more accurately obtain the changed area of ​​the parking scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic flow chart of the steps of the parking scene change detection method provided by the present invention.

[0055] Figure 2 It is a structural diagram of the parking scene change detection system provided by the present invention. DETAILED DESCRIPTION

[0056] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments.

[0057] See Figure 1 and Figure 2 The present invention provides a parking scene change detection method, comprising:

[0058] S10, constructing a parking scene dataset, the parking scene dataset including multiple groups of scene images, each group of scene images including a dual-phase scene image and a change label image;

[0059] S20, sequentially passing the dual-temporal scene image through the first layer network structure, the second layer network structure, and the third layer network structure of the scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map;

[0060] S30, upsampling the feature difference map and performing a jump connection with the third feature map to obtain a first jump connection feature, upsampling the first jump connection feature and performing a jump connection with the second feature map to obtain a second jump connection feature, upsampling the second jump connection feature and performing a jump connection with the first feature map to obtain a third jump connection feature;

[0061] S40, performing pixel-by-pixel classification on the third skip connection feature to generate a bi-temporal scene change region image, and optimizing parameters of a scene change detection model based on the bi-temporal scene change region image and the change label image to obtain a trained scene change detection model;

[0062] S50 , performing parking scene change detection on multiple groups of dual-temporal scene images to be tested according to the trained scene change detection model, and obtaining change area images corresponding to the dual-temporal scene images to be tested.

[0063] In this embodiment, in S10, the multi-temporal scene image is derived from the multi-temporal image or video data collected by the monitoring device of the parking space and its surroundings, and represents the image of the same geographical location at multiple times. The multi-temporal scene image can be understood as images at multiple times, i.e., t1, t2, ..., t n In one embodiment, the image includes, but is not limited to, multi-temporal images and dual-temporal images, which are uniformly described below as dual-temporal images. The change label image is a binary label image generated by transforming the change region. The dual-temporal scene image and the change label image form a set of scene images.

[0064] In S20, the dual-phase scene image is used as the input of the entire scene change detection model, and passes through the bottom, middle, and high first-layer network structure, the second-layer network structure, and the third-layer network structure of the network in sequence to extract multi-layer feature maps from different resolution dimensions. The extracted multi-layer feature maps not only contain the spatial detail information of the bottom layer but also have high-level semantic information. The scene change detection model can be a UNet network or a lightweight network such as EfficientNet, which is not specifically limited in this embodiment. During the feature extraction process, feature maps of different sizes are extracted from multiple layers by the scene change detection model. Multiple feature maps contain more image detail information and semantic feature information, forming a first feature map, a second feature map, a third feature map, and a feature difference map.

[0065] In S30, the feature difference map is upsampled and jump-connected with the third feature map, the first jump-connected feature is upsampled and jump-connected with the second feature map, and the second jump-connected feature is upsampled and jump-connected with the first feature map to obtain the first jump-connected feature, the second jump-connected feature and the third jump-connected feature, respectively. The output features of each layer of the network are fused at different scales through the jump connection method, thereby realizing feature extraction of various detail information of the image and better detecting the changed areas of the parking scene.

[0066] In S40, through steps S10 to S30, a third skip connection feature is ultimately obtained. This feature integrates the features extracted by each network layer of the scene change detection model, fully capturing the detailed feature information of the image and extracting high-level semantic features. The third skip connection feature is then classified pixel by pixel to determine whether each pixel belongs to a change category. This generates a bi-temporal scene change region image, and obtains the change region image predicted by the scene change detection model. Consequently, a loss function is constructed based on the bi-temporal scene change region image and the change label image, and the parameters of the scene change detection model are optimized and updated, ultimately obtaining a trained scene change detection model.

[0067] In S50, the dual-phase scene image to be tested is a dual-phase scene image that needs to be detected outside the parking scene dataset, and is input into the trained scene change detection model to detect the parking scene change area, thereby obtaining the change area image corresponding to the dual-phase scene image to be tested.

[0068] The parking scene change detection method provided by the present invention reduces the interference of pseudo-changes in parking scenes by constructing a parking scene dataset, marking the change areas of interest, and ignoring changes that are not of interest. The scene change detection model is used to learn the features of the dual-phase scene image, and through the low, medium and high three-layer network structure, the feature map of each layer and the feature difference map of the last layer are extracted, and step-by-step jump connection is implemented between the feature difference map and the multiple feature maps to obtain the third jump connection feature formed after the jump connection. The third jump connection feature is classified pixel by pixel to generate a dual-phase scene change area image predicted by the scene change detection model, and compared with the change label image annotated in the dataset to realize the training of the scene change detection model, which is used for the subsequent detection of the dual-phase scene image to be tested, and the corresponding change area image is obtained, thereby realizing the detection of the change area of ​​the parking scene.

[0069] Through the parking scene change detection method provided by the present invention, the dual-phase scene image undergoes feature extraction at multiple different network levels, fully learns the features of each detail in the image, and jumps the features extracted from each layer of the network structure to fully integrate the various feature details in the image, and performs pixel-by-pixel classification to obtain the changed area image, reducing the influence of external factors such as light intensity, background fluctuations, and shadows. It can better solve the problem of low change detection accuracy caused by diverse and unclear image changes, and can more accurately obtain the changed area of ​​the parking scene.

[0070] In one embodiment, S10 is to construct a parking scene dataset, where the parking scene dataset includes multiple sets of scene images, each set of scene images including a bi-temporal scene image and a change label image, including:

[0071] S110, acquiring multi-temporal scene images of multiple different regions, and reconstructing multiple dual-temporal scene images of each region based on the multi-temporal scene images of each region, marking the change position of each dual-temporal scene image, and obtaining the change region;

[0072] S120, converting the changed region into a binary label image to obtain a changed label image;

[0073] S130 , converting the plurality of dual-temporal scene images and the corresponding change label images into images of the same size to construct a parking scene dataset.

[0074] In this embodiment, a place (which can also be understood as an area) corresponds to a multi-temporal scene image, and multiple different areas can be understood as multiple different places. The parking scene contains a variety of objects and other information. According to the application requirements of the parking scene, customize which areas in the multi-temporal scene image are change areas and which are pseudo-change areas. By marking the change areas, it is possible to focus on learning the change areas, and it is particularly important to reduce the interference of pseudo-changes. Convert the change area to generate a binary label image to obtain a change label image. Convert the dual-phase scene image and the corresponding change label image into an image group of the same size to form a parking scene dataset.

[0075] In one embodiment, S20, the dual-temporal scene image is sequentially passed through the first network structure, the second network structure, and the third network structure of the scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map, including:

[0076] S210, inputting the dual-temporal scene image into the first layer network structure, outputting a first low-level feature map and a second low-level feature map, and performing feature fusion on the first low-level feature map and the second low-level feature map according to an attention mechanism method to obtain a first feature map;

[0077] S220: Input the first low-level feature map and the second low-level feature map into the second-layer network structure, output a first intermediate feature map and a second intermediate feature map, and perform feature fusion on the first intermediate feature map and the second intermediate feature map according to the attention mechanism method to obtain a second feature map;

[0078] S230: Input the first intermediate feature map and the second intermediate feature map into the third layer network structure, output the first high-level feature map and the second high-level feature map, and perform feature fusion on the first high-level feature map and the second high-level feature map according to the attention mechanism method to obtain a third feature map;

[0079] S240: Calculate the feature difference between the first high-level feature map and the second high-level feature map to obtain a feature difference map.

[0080] In this embodiment, a multi-temporal scene image is fed through a first-layer network structure to output two low-level feature maps. An attention mechanism is then used to fuse these two low-level feature maps to obtain a first feature map, thereby obtaining a new feature map. During the feature fusion process, attention mechanisms include, but are not limited to, pixel-level attention mechanisms, channel-level attention mechanisms, or spatial attention mechanisms. By further fusing these two low-level feature maps, attention to the spatial and temporal feature differences of the dual-temporal scene image is enhanced, enabling better learning of the features of the multi-temporal scene image.

[0081] The low-level feature maps of the bi-temporal scene images are fed into the second-layer network structure to output two mid-level feature maps. These are then fused using an attention mechanism to generate a second feature map, yielding a new feature map. This further fusion of the two mid-level feature maps enhances attention to the temporal, spatial, and semantic differences in the bi-temporal scene images, enabling better learning of the features of multi-temporal scene images.

[0082] The intermediate feature images of the bi-temporal scene graph are fed into the third-layer network structure to output two high-level feature maps. These two high-level feature maps are then fused using an attention mechanism to generate a third feature map, yielding a new feature map. This further fusion of the two high-level feature maps enhances attention to the semantic differences in the bi-temporal scene images, enabling better learning of the features of multi-temporal scene images.

[0083] During feature extraction, the first and second high-level feature maps extracted from the third-layer network structure are used to calculate feature differences between the two high-level feature maps to obtain a feature difference map, which is then used to construct a new feature map. Methods such as feature absolute value difference can be used to calculate feature differences, but this embodiment is not limited to this method.

[0084] After the dual-temporal scene image passes through the first, second, and third network layers of the scene change detection model and undergoes feature fusion using the attention mechanism, the fused first, second, and third feature maps, as well as a feature difference map, are obtained. The attention mechanism enhances focus on the difference regions in the output features of each layer of the network structure, thereby obtaining a change map corresponding to the multi-temporal scene image. The impact of class imbalance during pixel-by-pixel classification is mitigated in the loss function, further improving the detection accuracy of the parking scene change detection method of the present invention.

[0085] In one embodiment, S30, upsampling the feature difference map and performing a jump connection with the third feature map to obtain a first jump connection feature, upsampling the first jump connection feature and performing a jump connection with the second feature map to obtain a second jump connection feature, and upsampling the second jump connection feature and performing a jump connection with the first feature map to obtain a third jump connection feature, includes:

[0086] S310, performing upsampling processing on the feature difference map to obtain an upsampled difference map;

[0087] S320, inputting the upsampled difference map and the third feature map into a skip connection convolutional network to form a first skip connection feature;

[0088] S330, upsampling the first skip connection feature and inputting it and the second feature map into a skip connection convolutional network to form a second skip connection feature;

[0089] S340 , upsampling the second skip connection feature and inputting it and the first feature map into a skip connection convolutional network to form a third skip connection feature.

[0090] In this embodiment, further feature fusion between the feature difference map, the third feature map, the second feature map and the first feature map is achieved through the jump connection convolutional network, which weakens the impact of the gradient disappearance problem and enables better feature fusion.

[0091] In one embodiment, at S50, after performing parking scene change detection on a plurality of sets of dual-temporal scene images to be tested based on the trained scene change detection model and obtaining changed region images corresponding to the dual-temporal scene images to be tested, the method further includes:

[0092] S60, mapping the changed region image pixel by pixel to the parking scene image corresponding to the last moment in the dual-temporal scene image to be measured, to generate a changed mask region corresponding to the parking scene image;

[0093] S70 , obtaining a change direction and a change amount of the changed mask area relative to the berth area according to the changed mask area and the berth area in the parking scene image.

[0094] In this embodiment, the parking scene includes static parking spaces, dynamic vehicles, and other common objects. The change region image is mapped pixel by pixel onto the parking scene image at the last moment in the dual-temporal scene image to be measured, generating a change mask region. The change mask region and the parking space region in the image produce a change, indicating that the current parking space status has changed. The change can be expressed as:

[0095]

[0096] Where & represents the intersection of the change mask region and the berth region in the image. Area() represents the area of ​​the berth region. The change direction and amount of change in the berth region are obtained from the relative positions of the change mask region and the berth region. If a valid amount of change occurs in the berth region, the change direction is further obtained to determine whether a valid berth region change has occurred. If a change occurs in the berth region, the relative state of the multi-temporal scene image has changed.

[0097] The change mask area and the berth area of ​​the dual-temporal scene image reveal detailed changes in the berth. Changes in the berth status result in changes in the dual-temporal parking scene. Mapping the change mask area to the parking scene at the corresponding time period reveals the change status of each component within the parking scene. The parking scene change detection method provided by the present invention qualitatively and quantitatively analyzes whether parking scenes have changed at different times. The spatial location and size of the changed area within the scene are identified by the direction and amount of change. The change status of the parking scene, the change area, and the berth are then used to further assess and analyze the type of parking scene change, providing an accurate basis for precisely determining the parking scene status.

[0098] See Figure 2 As shown, the present invention provides a parking scene change detection system 100. The parking scene change detection system 100 includes a data construction module 10, a feature extraction module 20, a feature jump connection module 30, a change detection model formation module 40, and a change detection module 50. The data construction module 10 is used to construct a parking scene dataset, which includes multiple sets of scene images, each set of scene images including a bi-temporal scene image and a change label image. The feature extraction module 20 is used to sequentially pass the bi-temporal scene image through the first layer network structure, the second layer network structure, and the third layer network structure of the scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map.

[0099] The feature jump connection module 30 is used to jump-connect the feature difference map after upsampling it with the third feature map to obtain a first jump connection feature, jump-connect the first jump connection feature after upsampling it with the second feature map to obtain a second jump connection feature, and jump-connect the second jump connection feature after upsampling it with the first feature map to obtain a third jump connection feature. The change detection model formation module 40 is used to classify the third jump connection feature pixel by pixel to generate a dual-phase scene change area image, optimize the parameters of the scene change detection model based on the dual-phase scene change area image and the change label image, and obtain a trained scene change detection model. The change detection module 50 is used to perform parking scene change detection on multiple groups of dual-phase scene images to be tested based on the trained scene change detection model to obtain change area images corresponding to the dual-phase scene images to be tested.

[0100] In this embodiment, the description of the data construction module 10 may refer to the description of S10 in the above embodiment. The description of the feature extraction module 20 may refer to the description of S20 in the above embodiment. The description of the feature jump connection module 30 may refer to the description of S30 in the above embodiment. The description of the change detection model formation module 40 may refer to the description of S40 in the above embodiment. The description of the change detection module 50 may refer to the description of S50 in the above embodiment.

[0101] In one embodiment, the feature extraction module 20 includes a first feature acquisition module, a second feature acquisition module, a third feature acquisition module, and a feature difference acquisition module. The first feature acquisition module is used to input the dual-phase scene image into the first layer network structure, output a first low-level feature map and a second low-level feature map, and perform feature fusion on the first low-level feature map and the second low-level feature map according to the attention mechanism method to obtain a first feature map. The second feature acquisition module is used to input the first low-level feature map and the second low-level feature map into the second layer network structure, output a first intermediate feature map and a second intermediate feature map, and perform feature fusion on the first intermediate feature map and the second intermediate feature map according to the attention mechanism method to obtain a second feature map.

[0102] The third feature acquisition module is used to input the first intermediate feature map and the second intermediate feature map into the third layer network structure, output the first high-level feature map and the second high-level feature map, and perform feature fusion on the first high-level feature map and the second high-level feature map according to the attention mechanism method to obtain the third feature map; the feature difference acquisition module is used to calculate the feature difference between the first high-level feature map and the second high-level feature map to obtain a feature difference map.

[0103] In this embodiment, the description of the first feature acquisition module can refer to the description of S210 in the above embodiment. The description of the second feature acquisition module can refer to the description of S220 in the above embodiment. The description of the third feature acquisition module can refer to the description of S230 in the above embodiment. The description of the feature difference acquisition module can refer to the description of S240 in the above embodiment.

[0104] In one embodiment, the feature jump connection module 30 includes an upsampling module, a first jump connection feature acquisition module, a second jump connection feature acquisition module, and a third jump connection feature acquisition module. The upsampling module is used to upsample the feature difference map to obtain an upsampled difference map. The first jump connection feature acquisition module is used to input the upsampled difference map and the third feature map into the jump connection convolutional network to form a first jump connection feature. The second jump connection feature acquisition module is used to input the first jump connection feature and the second feature map into the jump connection convolutional network to form a second jump connection feature. The third jump connection feature acquisition module is used to input the second jump connection feature and the first feature map into the jump connection convolutional network to form a third jump connection feature.

[0105] In this embodiment, the description of the upsampling module can refer to the description of S310 in the above embodiment. The description of the first skip connection feature acquisition module can refer to the description of S320 in the above embodiment. The description of the second skip connection feature acquisition module can refer to the description of S330 in the above embodiment. The description of the third skip connection feature acquisition module can refer to the description of S340 in the above embodiment.

[0106] In one embodiment, the parking scene change detection system 100 further includes a change mask region generation module and a change acquisition module. The change mask region generation module is configured to map the change region image pixel by pixel onto the parking scene image corresponding to the last moment in the dual-temporal scene image to be detected, thereby generating a change mask region corresponding to the parking scene image. The change acquisition module is configured to obtain the change direction and amount of change of the change mask region relative to the parking space region based on the change mask region and the parking space region in the parking scene image.

[0107] In this embodiment, the description of the change mask region generating module can refer to the description of S60 in the above embodiment. The description of the change acquiring module can refer to the description of S70 in the above embodiment.

[0108] In one embodiment, the data construction module 10 includes a labeling module, a change label image acquisition module, and a size change module. The labeling module is used to acquire multi-temporal scene images of multiple different regions, and based on the multi-temporal scene images of each region, reconstruct multiple dual-temporal scene images of each region, mark the change position of each dual-temporal scene image, and obtain the change region. The change label image acquisition module is used to convert the change region into a binary label image to obtain a change label image. The size change module is used to convert multiple dual-temporal scene images and corresponding change label images into images of the same size to construct a parking scene dataset.

[0109] In this embodiment, the description of the labeling module can refer to the description of S110 in the above embodiment. The description of the change label image acquisition module can refer to the description of S120 in the above embodiment. The description of the size change module can refer to the description of S130 in the above embodiment.

[0110] In the various embodiments described above, the specific order or hierarchy of steps in the disclosed processes is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of this disclosure. The accompanying method claims provide elements of the various steps in an exemplary order and are not intended to be limited to a specific order or hierarchy.

[0111] Those skilled in the art will also appreciate that the various illustrative logical blocks, modules, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of the two. To clearly demonstrate the interchangeability of hardware and software, the various illustrative components, modules, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0112] The various illustrative logic blocks or modules described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0113] The steps of the methods or algorithms described in the embodiments of the present invention may be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, the storage medium may be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium may also be integrated into the processor. The processor and storage medium may be provided in an ASIC, which may be provided in a user terminal. Alternatively, the processor and storage medium may also be provided in different components in the user terminal.

[0114] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A parking scene change detection method, characterized in that: include: Constructing a parking scene dataset, the parking scene dataset including multiple groups of scene images, each group of scene images including a bi-temporal scene image and a change label image; Passing the dual-phase scene image sequentially through the first layer network structure, the second layer network structure, and the third layer network structure of the scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map; Upsampling the feature difference map and performing a jump connection with the third feature map to obtain a first jump connection feature, upsampling the first jump connection feature and performing a jump connection with the second feature map to obtain a second jump connection feature, and upsampling the second jump connection feature and performing a jump connection with the first feature map to obtain a third jump connection feature; performing pixel-by-pixel classification on the third skip connection feature to generate a bi-temporal scene change region image, and optimizing parameters of the scene change detection model based on the bi-temporal scene change region image and the change label image to obtain a trained scene change detection model; Performing parking scene change detection on a plurality of groups of dual-phase scene images to be tested according to the trained scene change detection model to obtain change area images corresponding to the dual-phase scene images to be tested; sequentially passing the dual-phase scene images through the first-layer network structure, the second-layer network structure, and the third-layer network structure of the scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map, including: Inputting the bi-temporal scene image into the first-layer network structure, outputting a first low-level feature map and a second low-level feature map, and performing feature fusion on the first low-level feature map and the second low-level feature map according to an attention mechanism method to obtain the first feature map; Inputting the first low-level feature map and the second low-level feature map into the second-layer network structure, outputting a first intermediate feature map and a second intermediate feature map, and performing feature fusion on the first intermediate feature map and the second intermediate feature map according to the attention mechanism method to obtain the second feature map; Inputting the first intermediate feature map and the second intermediate feature map into the third-layer network structure, outputting a first high-level feature map and a second high-level feature map, and performing feature fusion on the first high-level feature map and the second high-level feature map according to the attention mechanism method to obtain the third feature map; Calculate the feature difference between the first high-level feature map and the second high-level feature map to obtain the feature difference map.

2. The parking scene change detection method according to claim 1, characterized in that: The step of upsampling the feature difference map and performing a jump connection with the third feature map to obtain a first jump connection feature, upsampling the first jump connection feature and performing a jump connection with the second feature map to obtain a second jump connection feature, and upsampling the second jump connection feature and performing a jump connection with the first feature map to obtain a third jump connection feature includes: Performing upsampling processing on the feature difference map to obtain an upsampled difference map; Inputting the upsampled difference map and the third feature map into a skip connection convolutional network to form the first skip connection feature; Upsampling the first skip connection feature and inputting it and the second feature map into the skip connection convolutional network to form the second skip connection feature; The second skip connection feature is upsampled and inputted into the skip connection convolutional network together with the first feature map to form the third skip connection feature.

3. The parking scene change detection method according to claim 1, characterized in that: After performing parking scene change detection on a plurality of sets of dual-temporal scene images to be tested based on the trained scene change detection model and obtaining change area images corresponding to the dual-temporal scene images to be tested, the method further includes: Mapping the change region image pixel by pixel to the parking scene image corresponding to the last moment in the dual-temporal scene image to be measured, to generate a change mask region corresponding to the parking scene image; According to the change mask area and the berth area in the parking scene image, a change direction and a change amount of the change mask area relative to the berth area are obtained.

4. The parking scene change detection method according to claim 1, characterized in that: The parking scene dataset is constructed, wherein the parking scene dataset includes multiple groups of scene images, each group of scene images includes a dual-phase scene image and a change label image, including: Acquire multiple multi-temporal scene images of different regions, and reconstruct multiple dual-temporal scene images of each region based on the multi-temporal scene images of each region, mark the change position of each dual-temporal scene image, and obtain the change region; Converting the changed region into a binary label image to obtain the changed label image; The plurality of dual-phase scene images and the corresponding change label images are converted into images of the same size to construct the parking scene dataset.

5. A parking scene change detection system, characterized in that: include: A data construction module is used to construct a parking scene dataset, wherein the parking scene dataset includes multiple groups of scene images, each group of scene images includes a dual-phase scene image and a change label image; a feature extraction module, configured to sequentially pass the dual-temporal scene image through a first-layer network structure, a second-layer network structure, and a third-layer network structure of a scene change detection model to obtain a first feature map, a second feature map, a third feature map, and a feature difference map; a feature skip connection module, configured to perform a skip connection on the feature difference map after upsampling and the third feature map to obtain a first skip connection feature, perform a skip connection on the first skip connection feature after upsampling and the second feature map to obtain a second skip connection feature, and perform a skip connection on the second skip connection feature after upsampling and the first feature map to obtain a third skip connection feature; a change detection model forming module, configured to perform pixel-by-pixel classification on the third skip connection feature to generate a bi-temporal scene change region image, and optimize parameters of the scene change detection model based on the bi-temporal scene change region image and the change label image to obtain a trained scene change detection model; a change detection module, configured to perform parking scene change detection on a plurality of sets of dual-temporal scene images to be tested based on the trained scene change detection model, and obtain change area images corresponding to the dual-temporal scene images to be tested; The feature extraction module includes: A first feature acquisition module is configured to input the bi-temporal scene image into the first-layer network structure, output a first low-level feature map and a second low-level feature map, and perform feature fusion on the first low-level feature map and the second low-level feature map according to an attention mechanism method to obtain the first feature map; A second feature acquisition module is configured to input the first low-level feature map and the second low-level feature map into the second layer network structure, output a first intermediate feature map and a second intermediate feature map, and perform feature fusion on the first intermediate feature map and the second intermediate feature map according to the attention mechanism method to obtain the second feature map; A third feature acquisition module is configured to input the first intermediate feature map and the second intermediate feature map into the third layer network structure, output a first high-level feature map and a second high-level feature map, and perform feature fusion on the first high-level feature map and the second high-level feature map according to the attention mechanism method to obtain the third feature map; The feature difference acquisition module is used to calculate the feature difference between the first high-level feature map and the second high-level feature map to obtain the feature difference map.

6. The parking scene change detection system according to claim 5, characterized in that: The feature skip connection module includes: an upsampling module, configured to perform upsampling processing on the feature difference map to obtain an upsampled difference map; a first skip connection feature acquisition module, configured to input the upsampled difference map and the third feature map into a skip connection convolutional network to form the first skip connection feature; a second skip connection feature acquisition module, configured to upsample the first skip connection feature and input the first skip connection feature and the second feature map into the skip connection convolutional network to form the second skip connection feature; The third skip connection feature acquisition module is used to input the second skip connection feature and the first feature map into the skip connection convolutional network after upsampling to form the third skip connection feature.

7. The parking scene change detection system according to claim 5, characterized in that: The system further comprises: a change mask region generating module, configured to map the change region image pixel by pixel to the parking scene image corresponding to the last moment in the dual-temporal scene image to be measured, and generate a change mask region corresponding to the parking scene image; The change acquisition module is used to obtain the change direction and change amount of the change mask area relative to the berth area based on the change mask area and the berth area in the parking scene image.

8. The parking scene change detection system according to claim 5, characterized in that: The data construction module includes: a marking module, configured to acquire multi-temporal scene images of a plurality of different regions, and reconstruct a plurality of dual-temporal scene images of each region based on the multi-temporal scene images of each region, and mark a change position of each dual-temporal scene image to obtain a change region; a change label image acquisition module, configured to convert the change region into a binary label image to obtain the change label image; The size change module is used to convert the plurality of the dual-phase scene images and the corresponding change label images into images of the same size to construct the parking scene dataset.

Images