Automatic identification method and device for vehicle condition damage based on visual image

Abstract

Embodiments of the present disclosure disclose a visual image-based automatic vehicle damage identification method and device. A specific embodiment of the method comprises: in response to a driving trend of a target vehicle being a driving-away trend, collecting a local vehicle body image group sequence by at least three cameras arranged on a gate barrier; generating a spliced vehicle body image sequence according to the local vehicle body image group sequence; identifying vehicle body damage in the spliced vehicle body image in the spliced vehicle body image sequence to generate a current vehicle body damage information set; reading a historical vehicle body damage information set matched with vehicle identity information corresponding to the target vehicle; determining whether there is target vehicle body damage information; and in response to there being target vehicle body damage information, pushing the target vehicle body damage information to a user terminal bound with the target vehicle. The embodiment realizes effective evidence collection of vehicle body damage generated during parking of the vehicle.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, particularly the field of artificial intelligence technology, and specifically to a method and apparatus for automatic vehicle damage identification based on visual images. Background Technology

[0002] With the continuous increase in car ownership, the problem of vehicle damage is on the rise. Especially during parking, the environment of the parking location (e.g., poor lighting in a garage) makes vehicles highly susceptible to damage. Although electric vehicles now incorporate trigger-based monitoring technologies such as "sentinel mode," a large number of vehicles still lack similar functionality. This results in damage occurring during parking going undetected, and even when detected, it's difficult to effectively trace the location of the damage, thus hindering the collection of effective evidence. Summary of the Invention

[0003] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0004] Some embodiments of this disclosure propose a method and apparatus for automatic vehicle damage identification based on visual images to solve the technical problems mentioned in the background section above.

[0005] In a first aspect, some embodiments of this disclosure provide an automatic vehicle damage recognition method based on visual images. The method includes: responding to a target vehicle's driving trend being a departure trend, acquiring vehicle body images of the target vehicle using at least three cameras installed on a gate barrier, obtaining a sequence of partial vehicle body images, wherein the target vehicle is the vehicle that triggered the inductive loop inside the gate barrier, and the partial vehicle body image sets correspond to the cameras; generating a stitched vehicle body image sequence based on the partial vehicle body image sequence; and performing vehicle body damage recognition on the stitched vehicle body images in the stitched vehicle body image sequence to generate a current vehicle body damage information set, wherein the current vehicle... The vehicle damage information includes: damage type, damage feature vector, vehicle side identification, relative damage location, and damage confidence level; it reads a set of historical vehicle damage information that matches the vehicle identity information corresponding to the target vehicle, wherein the set of historical vehicle damage information is generated after the target vehicle triggers the inductive loop coil outside the gate; based on the current set of vehicle damage information and the set of historical vehicle damage information, it determines whether target vehicle damage information exists, wherein the target vehicle damage information represents newly generated vehicle damage to the target vehicle; in response to the existence of target vehicle damage information, it pushes the target vehicle damage information to the user terminal bound to the target vehicle.

[0006] Secondly, some embodiments of this disclosure provide an automatic vehicle damage recognition device based on visual images. The device includes: a vehicle body image acquisition unit configured to, in response to a target vehicle's driving trend being a departure trend, acquire vehicle body images of the target vehicle using at least three cameras installed on a gate-type barrier, obtaining a sequence of partial vehicle body images, wherein the target vehicle is the vehicle that triggers the inductive loop coil inside the gate-type barrier, and the partial vehicle body image sets correspond to the cameras; a generation unit configured to generate a stitched vehicle body image sequence based on the partial vehicle body image sequence; and a vehicle body damage recognition unit configured to perform vehicle body damage recognition on the stitched vehicle body images in the stitched vehicle body image sequence to generate a current vehicle body damage information set. The current vehicle body damage information includes: damage type, damage feature vector, vehicle side identification, relative damage location, and damage confidence level; the reading unit is configured to read a set of historical vehicle body damage information that matches the vehicle identity information corresponding to the target vehicle, wherein the set of historical vehicle body damage information is generated after the target vehicle triggers the inductive loop coil outside the gate; the determining unit is configured to determine whether target vehicle body damage information exists based on the current vehicle body damage information set and the historical vehicle body damage information set, wherein the target vehicle body damage information represents newly generated vehicle body damage to the target vehicle; the pushing unit is configured to push the target vehicle body damage information to the user terminal bound to the target vehicle in response to the existence of target vehicle body damage information.

[0007] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0008] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0009] The above-described embodiments of this disclosure have the following beneficial effects: The automatic vehicle damage identification method based on visual images, as described in some embodiments of this disclosure, effectively achieves evidence collection of vehicle body damage generated during parking. Specifically, the reason for the inability to effectively collect evidence is that during parking, the vehicle is easily damaged due to the environmental influence of the parking location (e.g., dim lighting in the garage, narrow parking spaces, etc.). This damage can easily occur (e.g., scraping against walls or pillars, scraping caused by a neighboring vehicle opening its door, scraping caused by a neighboring vehicle entering or exiting, etc.). Based on this, the automatic vehicle damage identification method based on visual images, as described in some embodiments of this disclosure, firstly, responds to the target vehicle's driving trend being a departure trend, by using at least three cameras installed on the gate-type barrier to acquire vehicle body images of the target vehicle, obtaining a sequence of partial vehicle body images. The target vehicle is the vehicle that triggers the inductive loop inside the gate-type barrier, and the partial vehicle body image sequence corresponds to the camera. In practice, for vehicles parked in garages, gate barriers are often installed at the garage entrances and exits to control vehicle entry and exit. Therefore, this disclosure uses at least three cameras installed on the gate barrier to capture vehicle images of departing vehicles. Specifically, vehicle images are also captured when the target vehicle enters the garage. Next, based on the aforementioned local vehicle image sequence, a stitched vehicle image sequence is generated. In practice, due to differences in vehicle length and image acquisition range, this disclosure uses image stitching to form a holistic, stitched vehicle image. Then, vehicle damage identification is performed on the stitched vehicle images in the aforementioned stitched vehicle image sequence to generate a current vehicle damage information set, which includes: damage type, damage feature vector, vehicle side markings, relative damage location, and damage confidence. The location and identification of vehicle damage are achieved by combining the stitched vehicle images. Furthermore, the system reads a set of historical vehicle damage information that matches the vehicle identity information corresponding to the target vehicle. This set of historical vehicle damage information is generated after the target vehicle triggers the inductive loop detector outside the gate. Additionally, based on the current set of vehicle damage information and the historical set of vehicle damage information, it determines whether target vehicle damage information exists, where target vehicle damage information represents newly generated vehicle damage. By comparing vehicle damage before and after entering and exiting the gate, it is determined whether new vehicle damage (target vehicle damage information) occurred during parking. Finally, in response to the existence of target vehicle damage information, this information is pushed to the user terminal bound to the target vehicle. Newly generated vehicle damage is also pushed to the user terminal as a notification. This method effectively collects evidence of vehicle damage generated during parking. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0011] Figure 1 This is a flowchart of some embodiments of the automatic vehicle damage identification method based on visual images according to the present disclosure; Figure 2 This is a schematic diagram showing the positional relationship between the gate-type barrier and the inductive loop detector; Figure 3 This is a schematic diagram showing the location of the region of interest; Figure 4 This is a schematic diagram illustrating the process of generating information about damage to the front of the vehicle. Figure 5 This is a schematic diagram of the network structure of the backbone image feature extraction network; Figure 6 This is a schematic diagram of the structure of some embodiments of the automatic vehicle damage recognition device based on visual images according to the present disclosure; Figure 7 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0012] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0013] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0014] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0015] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0016] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0017] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] refer to Figure 1 The diagram illustrates a flow 100 of some embodiments of a visual image-based automatic vehicle damage identification method according to the present disclosure. This visual image-based automatic vehicle damage identification method includes the following steps: Step 101: In response to the target vehicle's driving trend being a departure trend, at least three cameras installed on the gate barrier are used to acquire vehicle body images of the target vehicle, resulting in a sequence of partial vehicle body images.

[0019] In some embodiments, the execution subject (e.g., a computing device) of the automatic vehicle damage identification method based on visual images may, in response to the target vehicle’s driving trend being a departure trend, acquire vehicle body images of the target vehicle through at least three cameras installed on the gate barrier to obtain a sequence of local vehicle body images.

[0020] The target vehicle is the vehicle that triggers the inductive loop detector inside the gate. The gate is a gate-type barrier used to control vehicle access. At least three cameras are installed inside the gate. Specifically, these cameras are embedded at the top and both ends of the gate's inner side. Inductive loop detectors are installed at the front and rear of the gate. When a vehicle enters the range of these loop detectors, it triggers the at least three cameras on the gate to capture images, and subsequently performs actions including but not limited to payment prompts, gate opening, and gate closing. In addition, the gate may also include gate arms, a control module, a power supply module, and other functional modules to ensure normal operation; these are not limited here. A partial vehicle image group corresponds to a camera. That is, the partial vehicle images in the partial vehicle image group are partial vehicle images captured by the same camera.

[0021] In practice, when a vehicle triggers the inductive loop detector inside the gate, it indicates that the vehicle is moving away. When a vehicle triggers the inductive loop detector outside the gate, it indicates that the vehicle is moving in. In particular, when a vehicle triggers either the inductive loop detector inside or outside the gate, at least three cameras will be activated to capture images of the vehicle body.

[0022] As an example, see Figure 2 The diagram showing the positional relationship between the gantry gate and the inductive loop detector is particularly important. Figure 2The diagram shows the positional relationship between the gantry gate and the inductive loop detectors from a top-down perspective. Inductive loop detectors A and B are installed on either side of the gantry gate, with gate A on the same side as the vehicle's entry direction and gate B on the same side as the vehicle's exit direction.

[0023] As another example, the gate-type barrier is equipped with at least three cameras: camera C1, camera C2, and camera C3. The resulting sequence of partial vehicle body images can then include: partial vehicle body image group I1, partial vehicle body image group I2, and partial vehicle body image group I3. Partial vehicle body image group I1 is the image group acquired by camera C1. Partial vehicle body image group I2 is the image group acquired by camera C2. Partial vehicle body image group I3 is the image group acquired by camera C3.

[0024] It should be noted that the aforementioned computing devices can be either hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed on the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0025] Step 102: Generate a stitched vehicle body image sequence based on the local vehicle body image group sequence.

[0026] In some embodiments, the aforementioned execution entity can generate a stitched vehicle body image sequence based on a sequence of local vehicle body image groups.

[0027] The stitched vehicle body image is obtained by stitching together local vehicle body images from a group of local vehicle body images.

[0028] In practice, due to the varying lengths of different vehicle models, it's difficult to capture the entire vehicle in a single image when using cameras for image acquisition. Therefore, multiple images need to be captured. However, image overlap is highly likely. If a sequence of partial vehicle images is directly used for subsequent vehicle damage identification, redundant feature extraction will occur due to the overlapping image portions, resulting in a waste of computational resources in the vehicle damage identification process. Therefore, image stitching methods such as feature point matching can be used.

[0029] In some optional implementations of certain embodiments, the execution entity generates a stitched vehicle body image sequence based on the aforementioned local vehicle body image group sequence, including: Step S1: For each local vehicle body image group in the above local vehicle body image group sequence, perform the following image stitching steps: Step S11: Within the region of interest, feature points are extracted from each of the aforementioned local vehicle body images in the local vehicle body image group to obtain the feature point information corresponding to the local vehicle body image.

[0030] The aforementioned regions of interest include: a first region of interest and a second region of interest. The region of interest is a region within a local vehicle image used for feature point extraction, and the first region of interest and the second region of interest are set relative to each other.

[0031] The feature point information includes: the feature point vector corresponding to at least one feature point extracted from the first region of interest and the feature point vector corresponding to at least one feature point extracted from the second region of interest.

[0032] In practice, algorithms such as SIFT (Scale Invariant Feature Transform) can be used to extract feature points in the first and second regions of interest in the local vehicle image to obtain feature point information.

[0033] As an example, see Figure 3 The diagram shows the locations of the regions of interest, where the first and second regions of interest are positioned opposite each other. The first region of interest is perpendicular to the vehicle's direction of travel. The second region of interest is also perpendicular to the vehicle's direction of travel.

[0034] Step S12: In response to the fact that the above-mentioned partial vehicle body image group is an image captured by a camera installed on the first side of the above-mentioned gate barrier, or an image captured by a camera installed on the second side of the above-mentioned gate barrier, the partial vehicle body images in the above-mentioned partial vehicle body image group are stitched together according to the feature point information corresponding to the partial vehicle body images to obtain a stitched vehicle body image corresponding to the first side of the above-mentioned target vehicle, or a stitched vehicle body image corresponding to the second side of the above-mentioned target vehicle.

[0035] Specifically, when the (target) vehicle is traveling in the direction of the gate, the first side of the gate can be the left side. The second side of the gate can be the right side. Since the cameras on both sides of the gate face the side of the vehicle, and the left and right sides of the vehicle are generally symmetrical, the corresponding images can be stitched together using the same method during the stitching process.

[0036] In practice, firstly, the overlapping region of adjacent local vehicle body images can be determined by feature point matching. Then, using the overlapping region as the intersection area, the local vehicle body images in the group of local vehicle body images are stitched together.

[0037] Step S13: In response to the fact that the above-mentioned partial vehicle body image group is an image captured by a camera installed on the third side of the above-mentioned gate barrier, a segmentation position group is determined based on the above-mentioned partial vehicle body image group.

[0038] The segmentation location is the image location used to group the local vehicle body images containing the third side, fourth side, and fifth side of the target vehicle.

[0039] In practice, image recognition can be performed on each local vehicle body image in a group of local vehicle body images to determine whether the local vehicle body image contains segmentation locations that distinguish the third and fourth sides, as well as the fourth and fifth sides. Specifically, image recognition networks such as CNNs (Convolutional Neural Networks) and tri-classifiers can be used to perform image recognition on the local vehicle body images. The tri-classifier's classification results include: a category containing the distinguishing locations of the fourth and third sides, a category containing the distinguishing locations of the third and fifth sides, and other categories. Specifically, since the fourth and third sides, as well as the third and fifth sides, of the target vehicle are distinguished by the skeletal structure on the upper part of the vehicle body, the aforementioned image recognition network can be used to identify and classify whether the local vehicle body image contains distinguishing locations for the fourth and third sides, as well as the third and fifth sides, thereby obtaining a group of segmentation locations. In particular, see further... Figure 2 When the (target) vehicle is exiting towards the gate, the left side of the target vehicle is the first side. The right side of the target vehicle is the second side. The upper side of the target vehicle is the third side. The front side of the target vehicle is the fourth side. The rear side of the target vehicle is the fifth side.

[0040] As an example, taking a partial vehicle body image group I1 as an example, the partial vehicle body image group I1 may include: partial vehicle body image I1-1, partial vehicle body image I1-2, partial vehicle body image I1-3, partial vehicle body image I1-4, partial vehicle body image I1-5, and partial vehicle body image I1-6. Among them, partial vehicle body image I1-2 includes the fourth and third sides of the target vehicle. Partial vehicle body image I1-5 includes the third and fifth sides of the target vehicle. Therefore, the segmentation position may include: the image position of partial vehicle body image I1-2 within the partial vehicle body image group I1 and the image position of partial vehicle body image I1-5 within the partial vehicle body image group I1.

[0041] Step S14: Based on the above segmentation position group, the above local vehicle body image group is divided into image groups to obtain a first local vehicle body image group, a second local vehicle body image group, and a third local vehicle body image group.

[0042] Among them, the first partial vehicle body image group corresponds to the third side of the target vehicle, the second partial vehicle body image group corresponds to the fourth side of the target vehicle, and the third partial vehicle body image group corresponds to the fifth side of the target vehicle.

[0043] As an example, continuing the above example, the first partial vehicle body image group includes: partial vehicle body image I1-2, partial vehicle body image I1-3, partial vehicle body image I1-4, and partial vehicle body image I1-5. The second partial vehicle body image group includes: partial vehicle body image I1-1 and partial vehicle body image I1-2. The third partial vehicle body image group includes: partial vehicle body image I1-5 and partial vehicle body image I1-6.

[0044] Step S15: Based on the feature point information corresponding to the local vehicle body images, perform image stitching on the local vehicle body images in the first local vehicle body image group, the second local vehicle body image group, and the third local vehicle body image group respectively to obtain the stitched vehicle body image corresponding to the third side of the target vehicle, the stitched vehicle body image corresponding to the fourth side of the target vehicle, and the stitched vehicle body image corresponding to the fifth side of the target vehicle.

[0045] In practice, the image stitching methods for the partial vehicle images in the first partial vehicle image group, the second partial vehicle image group, and the third partial vehicle image group can all use feature point matching to determine the overlapping area of ​​adjacent partial vehicle images. Then, using the overlapping area as the intersection area, the adjacent partial vehicle images are stitched together to obtain the stitched vehicle image corresponding to the third side of the target vehicle, the stitched vehicle image corresponding to the fourth side of the target vehicle, and the stitched vehicle image corresponding to the fifth side of the target vehicle.

[0046] Specifically, the first, second, fourth, and fifth sides of a vehicle are the most common locations for vehicle body damage. Furthermore, the fourth and fifth sides of a vehicle often involve more vehicle components (e.g., low beam headlights, high beam headlights, front bumper, grille, hood, brake lights, reversing lights, rear bumper, tailgate, etc.). Therefore, this disclosure performs separate image grouping and stitching on different sides of the vehicle for subsequent identification of vehicle body damage based on different scenarios.

[0047] Step 103: Perform vehicle damage identification on the stitched vehicle images in the stitched vehicle image sequence to generate the current vehicle damage information set.

[0048] In some embodiments, the aforementioned execution entity may perform vehicle damage recognition on the stitched vehicle images in the stitched vehicle image sequence to generate a current vehicle damage information set.

[0049] The current vehicle body damage information includes: damage type, damage feature vector, vehicle side markings, relative damage location, and damage confidence. The damage type represents the type of damage to the (target) vehicle body. For example, damage types may include, but are not limited to, scratches, dents, and breaks. The damage feature vector can represent a one-dimensional feature vector corresponding to the damage location. Specifically, the damage feature vector can be represented using a 1×512 dimensional feature vector. The vehicle side markings represent the side of the vehicle body where the damage occurred. Since this disclosure mainly identifies vehicle body damage on the first, second, third, fourth, and fifth sides of the vehicle, the vehicle side markings can include: ["1", "2", "3", "4", "5"]. Wherein, vehicle side marking "1" corresponds to the first side of the vehicle. Vehicle side marking "2" corresponds to the second side of the vehicle. Vehicle side marking "3" corresponds to the third side of the vehicle. Vehicle side marking "4" corresponds to the fourth side of the vehicle. Vehicle side marking "5" corresponds to the fifth side of the vehicle. The relative damage location represents the position of the vehicle body damage relative to the corresponding side frame of the vehicle body. Specifically, the relative position of the vehicle body side can be characterized by four position offsets. Damage confidence represents the confidence level when the damage type corresponds to the vehicle body damage. The damage confidence value ranges from [0, 1].

[0050] In practice, models such as YOLO (You Only Look Once) can be used to identify vehicle damage in stitched vehicle images to obtain current vehicle damage information.

[0051] In some optional implementations of certain embodiments, the execution entity performs vehicle damage identification on the stitched vehicle images in the stitched vehicle image sequence to generate a current vehicle damage information set, including: Step S1: For each stitched vehicle body image in the above stitched vehicle body image sequence, perform the following vehicle body damage recognition steps: For details, see Figure 4 The diagram shows the process of generating front vehicle body damage information, and the following steps S11 to S19.

[0052] Step S11: Using the backbone image feature extraction network, perform shallow image feature extraction on the above-mentioned stitched vehicle body image to obtain shallow image features.

[0053] Since all the stitched vehicle images in the stitched vehicle image set are images of the target vehicle, this disclosure sets up a backbone image feature extraction network to perform general shallow image feature extraction on the stitched vehicle images, thereby obtaining the corresponding shallow image features.

[0054] As an example, see Figure 5The diagram shows the network structure of the backbone image feature extraction network. The backbone network includes: downsampling layers F1, F2, F3, F4, and F5; upsampling layers F6, F7, F8, F9, F10, F11, F12, and F13; four 1×1 convolutional layers; and convolutional blocks BL1, BL2, BL3, and BL4. First, the stitched vehicle image is downsampled using downsampling layers F1, F2, F3, F4, and F5. The image size of the stitched vehicle image is assumed to be H×W×C, where H, W, and C represent the image height, image width, and number of image channels, respectively. The feature dimensions output by downsampling layer F1 are H / 2×W / 2×C / 2, F2 is H / 4×W / 4×C / 4, F3 is H / 8×W / 8×C / 8, F4 is H / 16×W / 16×C / 16, and F5 is H / 32×W / 32×C / 32. This converts the stitched vehicle image into feature maps of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image size. In particular, considering that vehicle damage may include minor scratches and dents, deeper downsampling structures were not used during downsampling to avoid feature loss due to excessively deep downsampling layers. Next, the features output from downsampling layer F5 are upsampled and upscaled to H / 16×W / 16×C / 16 by upsampling layer F6, then superimposed with the features output from downsampling layer F4, and finally convolved by a 1×1 convolutional layer. The features output from upsampling layer F6 are upsampled and upscaled to H / 8×W / 8×C / 8 by upsampling layer F7, then superimposed with the features output from downsampling layer F3, and finally convolved by a 1×1 convolutional layer. The features output from upsampling layer F7 are upsampled and upscaled to H / 4×W / 4×C / 4 by upsampling layer F8, then superimposed with the features output from downsampling layer F2, and finally convolved by a 1×1 convolutional layer. The features output from upsampling layer F8 are upsampled and upscaled to H / 2×W / 2×C / 2 by upsampling layer F9, then superimposed with the features output from downsampling layer F1. By using 1×1 convolutional layers, the channel dimension is reduced (by half), thereby reducing the amount of feature processing required in subsequent convolutional blocks. Specifically, a ReLU activation function is applied after each 1×1 convolutional layer. For example, for the features resulting from the superposition of downsampling layer F4 and upsampling layer F6 (feature dimension H / 16×W / 16×C / 16), after convolution by the corresponding 1×1 convolutional layer, the output feature dimension is H / 16×W / 16×C / 32.For example, the features resulting from the superposition of downsampling layer F3 and upsampling layer F7 (feature dimension H / 8×W / 8×C / 8), after convolution processing by the corresponding 1×1 convolutional layer, yield an output feature dimension of H / 8×W / 8×C / 16. Similarly, the features resulting from the superposition of downsampling layer F2 and upsampling layer F8 (feature dimension H / 4×W / 4×C / 4), after convolution processing by the corresponding 1×1 convolutional layer, yield an output feature dimension of H / 4×W / 4×C / 8. Furthermore, the features resulting from the superposition of downsampling layer F1 and upsampling layer F9 (feature dimension H / 2×W / 2×C / 2), after convolution processing by the corresponding 1×1 convolutional layer, yield an output feature dimension of H / 2×W / 2×C / 4. Further convolution processing is performed on the features output from the corresponding 1×1 convolutional layers using convolutional blocks BL1, BL2, BL3, and BL4. Each of the convolutional blocks BL1, BL2, BL3, and BL4 comprises three parallel convolutional layers and one stitching layer (with kernel sizes of 1×1, 3×3, and 5×5, respectively). Finally, the features output by the convolutional blocks are stacked and upsampled through upsampling layers F10, F11, F12, and F13 to obtain a feature map (shallow image features) with the same image size as the stitched vehicle body image.

[0055] Step S12: In response to the first, second, or third side of the target vehicle corresponding to the stitched vehicle body image, generate deep image features based on the deep feature extraction network of the first image and the shallow image features.

[0056] Step S13: In response to the fourth or fifth side of the target vehicle corresponding to the above-mentioned stitched vehicle body image, generate deep image features based on the second image deep feature extraction network and the above-mentioned shallow image features.

[0057] The first image deep feature extraction network and the second image deep feature extraction network are set up in parallel.

[0058] In practice, such as Figure 4As shown, when shallow image features are input to the first deep image feature extraction network, they are not input to the second deep image feature extraction network. Conversely, when shallow image features are input to the second deep image feature extraction network, they are not input to the first deep image feature extraction network. Specifically, both the first and second deep image feature extraction networks use ResNet-50. The reason for not using a deeper network structure is to avoid feature loss that could affect the accuracy of small object classification, and to reduce the amount of feature processing. Furthermore, the reason for setting two ResNet-50 networks with the same structure as the first and second deep image feature extraction networks is that the first and second sides of the vehicle mainly correspond to the two sides of the vehicle body, and the third side corresponds to the top of the vehicle. These three sides of the vehicle body are continuous, so the first deep image feature extraction network is used to uniformly extract the corresponding deep image features. However, the fourth and fifth sides correspond to the front and rear sides of the vehicle, and the vehicle body structure is complex. To more accurately identify minor vehicle body damage, a separate second deep image feature extraction network is used.

[0059] Step S14: Generate vehicle body skeleton features using the vehicle body skeleton feature extraction network and the aforementioned shallow image features.

[0060] In practice, the vehicle body frame feature extraction network uses the VGG-16 model to combine shallow image features to extract the positional features related to the vehicle body frame of the target vehicle.

[0061] Step S15: Determine the region of interest based on the damage locator and the deep image features described above.

[0062] In practice, the damage locator uses an RPN (Region Proposal Network) layer to regress the region of interest based on deep image features.

[0063] Step S16: Based on the damage type classifier and the local features of the aforementioned damage region of interest in the aforementioned deep image features, generate the damage type and damage confidence of the current vehicle body damage information.

[0064] The vehicle body damage recognition model includes a backbone image feature extraction network, a first deep image feature extraction network, a second deep image feature extraction network, a vehicle body skeleton feature extraction network, a damage locator, and a damage type classifier.

[0065] The damage type classifier consists of multiple fully connected layers plus a SoftMax layer.

[0066] Step S17: Perform feature shaping on the local features of the above-mentioned damage interest region in the above-mentioned deep image features to obtain the damage feature vector included in the current vehicle body damage information.

[0067] In practice, at least one fully connected layer can be used to map the local features of the aforementioned damaged region of interest in the aforementioned deep image features into a 1×512 feature vector, which serves as the damaged feature vector.

[0068] Step S18: Determine the initial damage location based on the aforementioned region of interest.

[0069] In practice, the aforementioned executing entity can use the coordinates of the center point of the region of interest as the initial damage location.

[0070] Step S19: Based on the above-mentioned vehicle body frame features and the above-mentioned initial damage location, generate the relative damage location included in the current vehicle body damage information.

[0071] In practice, based on the features of the vehicle body frame, the position coordinates of the target vehicle's body frame in the stitched vehicle image can be obtained. Based on this, horizontal and vertical extensions are made with the initial damage location as the center, and the coordinates of the intersection of the two extension lines with the body frame are taken as the relative position of the damage.

[0072] Step 104: Read the set of historical vehicle body damage information that matches the vehicle identity information corresponding to the target vehicle.

[0073] In some embodiments, the aforementioned executing entity may read a set of historical vehicle body damage information that matches the vehicle identity information corresponding to the target vehicle.

[0074] The vehicle identification information is used to identify the target vehicle. For example, vehicle identification information may include: license plate number, vehicle type, and body color. The historical vehicle damage information set and the current vehicle damage information set corresponding to the target vehicle can be linked through the vehicle identification information to facilitate direct retrieval from the database later. The historical vehicle damage information set is generated after the target vehicle triggers the inductive loop detector outside the gate. The generation process is the same as that of the current vehicle damage information set, i.e., after triggering, steps 101 to 103 are executed to generate the historical vehicle damage information set, which will not be elaborated further here.

[0075] Step 105: Determine whether target vehicle damage information exists based on the current vehicle damage information set and the historical vehicle damage information set.

[0076] In some embodiments, the aforementioned executing entity may determine whether target vehicle damage information exists based on the current vehicle damage information set and the historical vehicle damage information set.

[0077] Among them, the target vehicle body damage information represents the newly generated vehicle body damage.

[0078] In practice, the existence of target vehicle body damage information can be determined using the difference operation. For example, target vehicle body damage information = current vehicle body damage information set - historical vehicle body damage information set, which means that the target vehicle body damage information belongs to the current vehicle body damage information set but does not belong to the historical vehicle body damage information set.

[0079] In some optional implementations of certain embodiments, the execution entity determines whether target vehicle body damage information exists based on the current vehicle body damage information set and the historical vehicle body damage information set, including: Step S1: Construct the initial cluster center set.

[0080] Among them, the initial cluster centers in the above-mentioned initial cluster center set correspond one-to-one with the current vehicle body damage information in the above-mentioned current vehicle body damage information set.

[0081] In practice, for each initial cluster center, a corresponding cluster feature vector can be constructed based on the current vehicle information. In particular, the damage type and vehicle side markings can be encoded using one-hot encoding, and the encoded features are concatenated with the damage feature vector and the relative position of the damage to form the cluster feature vector.

[0082] Step S2: Based on the initial cluster center set mentioned above, perform information clustering on the historical vehicle damage information in the historical vehicle damage information set mentioned above to obtain at least one information cluster.

[0083] In practice, the cluster center of historical vehicle body damage information within at least one information cluster can be determined by calculating cosine similarity. Specifically, the historical vehicle body damage information also needs to have its corresponding cluster feature vector constructed according to step S1 above to facilitate cosine similarity calculation.

[0084] Step S3: For each of the above-mentioned at least one information clusters, perform the following processing steps: Step S31: In response to the fact that the above information cluster only includes current vehicle body damage information, the current vehicle body damage information included in the above information cluster is determined as the target vehicle body damage information.

[0085] Step S32: In response to the above information cluster including historical vehicle body damage information, determine the damage change trend based on the historical vehicle body damage information and current vehicle body damage information included in the above information cluster.

[0086] In practice, damage trends can be obtained by comparing the relative positions of damage in historical vehicle body damage information with those in current vehicle body damage information. A change in position indicates the potential for new damage to occur on top of existing damage. Alternatively, the damage type can be compared; a change in damage type suggests the possibility of more severe damage on top of the existing damage. These trends can be represented by "0" and "1". A trend of "0" indicates minimal damage change, while a trend of "1" indicates more severe damage on top of the existing damage.

[0087] Step S33: In response to the damage change trend characterization that the current vehicle body damage information corresponds to a damage expansion trend, the current vehicle body damage information included in the above information cluster is determined as the above target vehicle body damage information.

[0088] In practice, this method has enabled the identification of newly generated vehicle body damage, as well as more severe vehicle body damage that arises on top of existing vehicle body damage.

[0089] Step 106: In response to the existence of target vehicle damage information, push the target vehicle damage information to the user terminal bound to the target vehicle.

[0090] In some embodiments, the aforementioned execution entity may push the target vehicle damage information to the user terminal bound to the target vehicle in response to the existence of target vehicle damage information.

[0091] The user terminal can be a mobile terminal that is bound to the account of the vehicle owner of the target vehicle.

[0092] In practice, a wireless push method can be used to push the target vehicle damage information to the user terminal bound to the target vehicle through the server.

[0093] In some optional implementations of some embodiments, the above method further includes: Step S1: In response to receiving the vehicle body damage confirmation initiated by the aforementioned user terminal, determine whether there is candidate vehicle body damage information.

[0094] Among them, the aforementioned candidate vehicle body damage information is vehicle body damage information that matches the aforementioned target vehicle body damage information from a pre-constructed vehicle body damage information database.

[0095] In practice, once the owner of the target vehicle confirms the damage to the vehicle body, they can initiate a damage confirmation process through their user terminal. Specifically, the damage is primarily caused by incidents such as minor collisions between vehicles, and since the damage is reciprocal, another vehicle may also suffer damage in a relatively specific location. Therefore, vehicle body damage matching can be used to identify candidate damage information that could potentially cause damage to the target vehicle.

[0096] Step S2: In response to the existence of candidate vehicle body damage information, determine whether there is a first surveillance video based on the vehicle parking records of the vehicle corresponding to the candidate vehicle body damage information and the vehicle parking records of the target vehicle.

[0097] The first surveillance video mentioned above is a video that simultaneously contains information about the target vehicle and the vehicle corresponding to the candidate vehicle body damage.

[0098] In practice, algorithms such as object detection (e.g., YOLO model) can be used to filter surveillance videos. When there is a video that contains both the target vehicle and the vehicle corresponding to the candidate vehicle body damage information, it is used as the first surveillance video.

[0099] Step S3: In response to the existence of the first surveillance video, generate first surveillance video information based on the first surveillance video.

[0100] The aforementioned first monitoring video information includes: the monitoring video address and the start and end times of the video.

[0101] In practice, the storage address of the first surveillance video on the server can be read, as well as the start and end times of the first surveillance video, which can be used as the video start and end times.

[0102] Step S4: Generate a vehicle body damage record based on the target vehicle body damage information, the candidate vehicle body damage information, the vehicle identity information corresponding to the target vehicle, the vehicle identity information corresponding to the candidate vehicle body damage information, and the first monitoring video information.

[0103] In practice, the target vehicle body damage information, the aforementioned candidate vehicle body damage information, the vehicle identity information corresponding to the aforementioned target vehicle, the vehicle identity information corresponding to the aforementioned candidate vehicle body damage information, and the first monitoring video information can be filled into the vehicle body damage record template to obtain the vehicle body damage record.

[0104] Step S5: Store the above vehicle damage records on the vehicle damage record blockchain.

[0105] Among them, the vehicle damage record blockchain can be a private blockchain used to store fixed vehicle damage records.

[0106] In some optional implementations of some embodiments, the above method further includes: Step S1: In response to the absence of candidate vehicle body damage information, retrieve the second monitoring video.

[0107] The second surveillance video is a surveillance video of the target vehicle's parking location during the period the vehicle was parked.

[0108] In practice, damage to the target vehicle may also be caused by its own reasons (such as the vehicle scraping against walls or pillars). Therefore, if the first monitoring video is not available, the second monitoring video can be retrieved from the server.

[0109] Step S2: Generate second monitoring video information based on the second monitoring video mentioned above.

[0110] The second monitoring video information also includes: the monitoring video address and the start and end times of the video.

[0111] In practice, the storage address of the second monitoring video on the server can be read, as well as the start and end times of the second monitoring video, which can be used as the video start and end times.

[0112] Step S3: Generate a vehicle damage record based on the above target vehicle damage information, the vehicle identity information corresponding to the above target vehicle, and the second monitoring video information.

[0113] In practice, the aforementioned implementing entity can fill the target vehicle damage information, the vehicle identity information corresponding to the target vehicle, and the second monitoring video information into the vehicle damage record template to obtain the vehicle damage record.

[0114] Step S4: Store the vehicle damage record on the vehicle damage record blockchain.

[0115] In some optional implementations of some embodiments, the above method further includes: Step S1: Generate a query link for the above vehicle body damage records.

[0116] In practice, the query link can be a short-lived, valid address link used for accessing vehicle damage records. This ensures that the vehicle owner corresponding to the target vehicle can remotely confirm access via a user terminal.

[0117] Step S2: Initiate liveness verification to the aforementioned user terminal.

[0118] In practice, since vehicle damage records involve surveillance video, it is necessary to verify the identity of the user of the information (the vehicle owner corresponding to the target vehicle) to ensure information security. Specifically, this can be achieved through methods such as facial recognition, where the vehicle owner corresponding to the target vehicle can be verified for liveness detection via a user terminal.

[0119] Step S3: In response to successful liveness verification, synchronize the above query link to the above user terminal.

[0120] In practice, the query link can be pushed to the user terminal via a server. Specifically, the query link is encrypted during transmission.

[0121] The above-described embodiments of this disclosure have the following beneficial effects: The automatic vehicle damage identification method based on visual images, as described in some embodiments of this disclosure, effectively achieves evidence collection of vehicle body damage generated during parking. Specifically, the reason for the inability to effectively collect evidence is that during parking, the vehicle is easily damaged due to the environmental influence of the parking location (e.g., dim lighting in the garage, narrow parking spaces, etc.). This damage can easily occur (e.g., scraping against walls or pillars, scraping caused by a neighboring vehicle opening its door, scraping caused by a neighboring vehicle entering or exiting, etc.). Based on this, the automatic vehicle damage identification method based on visual images, as described in some embodiments of this disclosure, firstly, responds to the target vehicle's driving trend being a departure trend, by using at least three cameras installed on the gate-type barrier to acquire vehicle body images of the target vehicle, obtaining a sequence of partial vehicle body images. The target vehicle is the vehicle that triggers the inductive loop inside the gate-type barrier, and the partial vehicle body image sequence corresponds to the camera. In practice, for vehicles parked in garages, gate barriers are often installed at the garage entrances and exits to control vehicle entry and exit. Therefore, this disclosure uses at least three cameras installed on the gate barrier to capture vehicle images of departing vehicles. Specifically, vehicle images are also captured when the target vehicle enters the garage. Next, based on the aforementioned local vehicle image sequence, a stitched vehicle image sequence is generated. In practice, due to differences in vehicle length and image acquisition range, this disclosure uses image stitching to form a holistic, stitched vehicle image. Then, vehicle damage identification is performed on the stitched vehicle images in the aforementioned stitched vehicle image sequence to generate a current vehicle damage information set, which includes: damage type, damage feature vector, vehicle side markings, relative damage location, and damage confidence. The location and identification of vehicle damage are achieved by combining the stitched vehicle images. Furthermore, the system reads a set of historical vehicle damage information that matches the vehicle identity information corresponding to the target vehicle. This set of historical vehicle damage information is generated after the target vehicle triggers the inductive loop detector outside the gate. Additionally, based on the current set of vehicle damage information and the historical set of vehicle damage information, it determines whether target vehicle damage information exists, where target vehicle damage information represents newly generated vehicle damage. By comparing vehicle damage before and after entering and exiting the gate, it is determined whether new vehicle damage (target vehicle damage information) occurred during parking. Finally, in response to the existence of target vehicle damage information, this information is pushed to the user terminal bound to the target vehicle. Newly generated vehicle damage is also pushed to the user terminal as a notification. This method effectively collects evidence of vehicle damage generated during parking.

[0122] Further reference Figure 6As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a vehicle damage automatic identification device based on visual images. These device embodiments are similar to... Figure 1 Corresponding to the method embodiments shown, this visual image-based automatic vehicle damage recognition device can be specifically applied to various electronic devices.

[0123] like Figure 6 As shown, a vehicle damage automatic recognition device 600 based on visual images in some embodiments includes: a vehicle body image acquisition unit 601, a generation unit 602, a vehicle body damage recognition unit 603, a reading unit 604, a determination unit 605, and a push unit 606. The vehicle body image acquisition unit 601 is configured to, in response to the driving trend of a target vehicle being a departure trend, acquire vehicle body images of the target vehicle through at least three cameras installed on a gate-type barrier, obtaining a sequence of partial vehicle body image groups. The target vehicle is the vehicle that triggers the inductive loop coil inside the gate-type barrier, and the partial vehicle body image groups correspond to the cameras. The generation unit 602 is configured to generate a stitched vehicle body image sequence based on the partial vehicle body image group sequence. The vehicle body damage recognition unit 603 is configured to perform vehicle body damage recognition on the stitched vehicle body images in the stitched vehicle body image sequence to generate a current vehicle body damage information set, wherein the current vehicle body damage information includes: damage type, damage characteristics, etc. The system includes: a signature vector, a vehicle side marking, the relative location of the damage, and a damage confidence level; a reading unit 604 configured to read a set of historical vehicle body damage information matching the vehicle identity information corresponding to the target vehicle, wherein the set of historical vehicle body damage information is generated after the target vehicle triggers the inductive loop coil outside the gate; a determining unit 605 configured to determine whether target vehicle body damage information exists based on the current set of vehicle body damage information and the set of historical vehicle body damage information, wherein the target vehicle body damage information represents newly generated vehicle body damage to the target vehicle; and a pushing unit 606 configured to push the target vehicle body damage information to a user terminal bound to the target vehicle in response to the existence of target vehicle body damage information.

[0124] It is understandable that the units described in the visual image-based automatic vehicle damage recognition device 600 are similar to those in the reference device. Figure 1 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to the visual image-based automatic vehicle damage recognition device 600 and the units contained therein, and will not be repeated here.

[0125] The following is for reference. Figure 7 It illustrates a schematic diagram of the structure of an electronic device (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. Figure 7The electronic device shown is merely an example and should not be construed as limiting the functionality or scope of the embodiments of this disclosure. Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The memory may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause the processor to perform any of the methods described above. The processor provides computational and control capabilities to support the operation of the entire computer device. The internal memory provides an environment for the execution of the computer program in the non-volatile storage medium; when executed by the processor, the computer program causes the processor to perform any of the methods described above. The network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the computer device to which the present disclosure is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0126] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0127] In one embodiment, the processor is configured to run a computer program stored in a memory to perform the following steps: In response to the target vehicle's driving trend being a departure trend, at least three cameras installed on the gate barrier are used to acquire vehicle body images of the target vehicle, obtaining a sequence of partial vehicle body images, wherein the target vehicle is the vehicle that triggered the inductive loop inside the gate barrier, and the partial vehicle body image sets correspond to the cameras; based on the partial vehicle body image sequence, a stitched vehicle body image sequence is generated; vehicle body damage is identified in the stitched vehicle body images in the stitched vehicle body image sequence to generate a current vehicle body damage information set, wherein the current vehicle body... Damage information includes: damage type, damage feature vector, vehicle side identification, relative location of damage, and damage confidence level; reads a set of historical vehicle body damage information that matches the vehicle identity information corresponding to the target vehicle, wherein the set of historical vehicle body damage information is generated after the target vehicle triggers the inductive loop coil outside the gate; determines whether target vehicle body damage information exists based on the current set of vehicle body damage information and the set of historical vehicle body damage information, wherein the target vehicle body damage information represents newly generated vehicle body damage to the target vehicle; in response to the existence of target vehicle body damage information, pushes the target vehicle body damage information to the user terminal bound to the target vehicle.

[0128] This disclosure also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can be referred to the various embodiments of the methods described above.

[0129] The aforementioned computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. Alternatively, the aforementioned computer-readable storage medium may be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0130] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0131] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for automatic vehicle damage identification based on visual images, characterized in that, include: In response to the target vehicle's driving trend being a departure trend, at least three cameras installed on the gate barrier are used to acquire vehicle body images of the target vehicle, resulting in a sequence of partial vehicle body images. The target vehicle is the vehicle that triggers the inductive loop inside the gate barrier, and the partial vehicle body image sequence corresponds to the camera. Based on the local vehicle body image group sequence, a stitched vehicle body image sequence is generated; Vehicle damage is identified in the stitched vehicle images in the stitched vehicle image sequence to generate a current vehicle damage information set, which includes: damage type, damage feature vector, vehicle side identification, relative damage location, and damage confidence. Read the set of historical vehicle body damage information that matches the vehicle identity information corresponding to the target vehicle, wherein the set of historical vehicle body damage information is generated after the target vehicle triggers the inductive loop outside the gate barrier. Based on the current vehicle body damage information set and the historical vehicle body damage information set, it is determined whether there is target vehicle body damage information, wherein the target vehicle body damage information represents newly generated vehicle body damage to the target vehicle. In response to the existence of target vehicle damage information, the target vehicle damage information is pushed to the user terminal bound to the target vehicle. The step of identifying vehicle damage in the stitched vehicle image sequence to generate a current vehicle damage information set includes: For each stitched vehicle body image in the stitched vehicle body image sequence, the following vehicle body damage identification steps are performed: The shallow image features are obtained by using the backbone image feature extraction network to extract shallow image features from the stitched vehicle image. In response to the first, second, or third side of the target vehicle corresponding to the stitched vehicle body image, deep image features are generated based on the deep feature extraction network of the first image and the shallow image features; In response to the fourth or fifth side of the target vehicle corresponding to the stitched vehicle body image, deep image features are generated based on the second image deep feature extraction network and the shallow image features, wherein the first image deep feature extraction network and the second image deep feature extraction network are set in parallel; The vehicle body skeleton features are generated by using the vehicle body skeleton feature extraction network and the shallow image features; Based on the damage locator and the deep image features, the region of interest for the damage is determined; Based on the damage type classifier and the local features of the region of interest in the deep image features, the damage type and damage confidence of the current vehicle body damage information are generated. The backbone image feature extraction network, the first deep image feature extraction network, the second deep image feature extraction network, the vehicle body skeleton feature extraction network, the damage locator, and the damage type classifier are included in the vehicle body damage recognition model. Feature shaping is performed on the local features of the region of interest in the damage in the deep image features to obtain the damage feature vector included in the current vehicle body damage information; The initial damage location is determined based on the region of interest. Based on the vehicle frame features and the initial damage location, the relative damage location is generated as part of the current vehicle damage information.

2. The method according to claim 1, characterized in that, The method further includes: In response to receiving a vehicle body damage confirmation initiated by the user terminal, it is determined whether there is candidate vehicle body damage information, wherein the candidate vehicle body damage information is vehicle body damage information that matches the target vehicle body damage information in a pre-constructed vehicle body damage information database; In response to the existence of candidate vehicle body damage information, based on the vehicle parking records of the vehicle corresponding to the candidate vehicle body damage information and the vehicle parking records of the target vehicle, it is determined whether there is a first monitoring video, wherein the first monitoring video is a video that simultaneously includes the target vehicle and the vehicle corresponding to the candidate vehicle body damage information. In response to the existence of a first surveillance video, first surveillance video information is generated based on the first surveillance video, wherein the first surveillance video information includes: surveillance video address and video start and end time; A vehicle body damage record is generated based on the target vehicle body damage information, the candidate vehicle body damage information, the vehicle identity information corresponding to the target vehicle, the vehicle identity information corresponding to the candidate vehicle body damage information, and the first monitoring video information. The vehicle body damage records are stored on the vehicle body damage record blockchain.

3. The method according to claim 2, characterized in that, The method further includes: In response to the absence of candidate vehicle body damage information, a second monitoring video is retrieved, wherein the second monitoring video is a monitoring video of the target vehicle at its parking position during the parking period; Based on the second surveillance video, generate second surveillance video information; A vehicle damage record is generated based on the target vehicle damage information, the vehicle identity information corresponding to the target vehicle, and the second surveillance video information. Vehicle damage records are stored on a vehicle damage record blockchain.

4. The method according to claim 3, characterized in that, The method further includes: Generate a query link for the vehicle body damage record; Initiate a liveness verification to the user terminal; Upon successful liveness verification, the query link is synchronized to the user terminal.

5. The method according to claim 4, characterized in that, The step of generating a stitched vehicle image sequence based on the local vehicle image group sequence includes: For each local vehicle body image group in the sequence of local vehicle body images, the following image stitching steps are performed: Within the region of interest, feature points are extracted from each local vehicle body image in the local vehicle body image group to obtain feature point information corresponding to the local vehicle body image. The region of interest includes a first region of interest and a second region of interest. The region of interest is a region within the local vehicle body image used for feature point extraction, and the first region of interest and the second region of interest are set relative to each other. In response to the fact that the partial vehicle body image group is an image captured by a camera set on the first side of the gate barrier or an image captured by a camera set on the second side of the gate barrier, the partial vehicle body images in the partial vehicle body image group are stitched together according to the feature point information corresponding to the partial vehicle body images to obtain a stitched vehicle body image corresponding to the first side of the target vehicle or a stitched vehicle body image corresponding to the second side of the target vehicle. In response to the fact that the partial vehicle body image group is an image captured by a camera installed on the third side of the gate, a segmentation position group is determined based on the partial vehicle body image group; Based on the segmented position group, the local vehicle body image group is divided into image groups to obtain a first local vehicle body image group, a second local vehicle body image group, and a third local vehicle body image group. The first local vehicle body image group corresponds to the third side of the target vehicle, the second local vehicle body image group corresponds to the fourth side of the target vehicle, and the third local vehicle body image group corresponds to the fifth side of the target vehicle. Based on the feature point information corresponding to the local vehicle body images, the local vehicle body images in the first local vehicle body image group, the second local vehicle body image group, and the third local vehicle body image group are stitched together to obtain the stitched vehicle body image corresponding to the third side of the target vehicle, the stitched vehicle body image corresponding to the fourth side of the target vehicle, and the stitched vehicle body image corresponding to the fifth side of the target vehicle.

6. The method according to claim 5, characterized in that, The step of determining whether target vehicle damage information exists based on the current vehicle damage information set and the historical vehicle damage information set includes: Construct an initial cluster center set, wherein the initial cluster centers in the initial cluster center set correspond one-to-one with the current vehicle body damage information in the current vehicle body damage information set; Based on the initial cluster center set, information clustering is performed on the historical vehicle damage information in the historical vehicle damage information set to obtain at least one information cluster; For each of the at least one information cluster, perform the following processing steps: In response to the fact that the information cluster only includes current vehicle body damage information, the current vehicle body damage information included in the information cluster is determined as the target vehicle body damage information; In response to the information cluster including historical vehicle body damage information, the damage change trend is determined based on the historical vehicle body damage information and current vehicle body damage information included in the information cluster; In response to the damage change trend characterizing the current vehicle body damage information as a trend of damage expansion, the current vehicle body damage information included in the information cluster is determined as the target vehicle body damage information.

7. An automatic vehicle damage identification device based on visual images, characterized in that, include: The vehicle body image acquisition unit is configured to, in response to the driving trend of the target vehicle being a departure trend, acquire vehicle body images of the target vehicle through at least three cameras installed on the gate barrier, and obtain a sequence of partial vehicle body images, wherein the target vehicle is the vehicle that triggers the ground induction coil inside the gate barrier, and the partial vehicle body image group corresponds to the camera; the generation unit is configured to generate a stitched vehicle body image sequence based on the sequence of partial vehicle body images. The vehicle body damage recognition unit is configured to perform vehicle body damage recognition on the stitched vehicle body images in the stitched vehicle body image sequence to generate a current vehicle body damage information set, wherein the current vehicle body damage information includes: damage type, damage feature vector, vehicle body side marking, damage relative position and damage confidence. The reading unit is configured to read a set of historical vehicle body damage information that matches the vehicle identity information corresponding to the target vehicle, wherein the set of historical vehicle body damage information is generated after the target vehicle triggers the inductive loop outside the gate barrier. The determining unit is configured to determine whether target vehicle body damage information exists based on the current vehicle body damage information set and the historical vehicle body damage information set, wherein the target vehicle body damage information represents newly generated vehicle body damage to the target vehicle. The push unit is configured to, in response to the existence of target vehicle damage information, push the target vehicle damage information to a user terminal bound to the target vehicle. The step of identifying vehicle damage in the stitched vehicle image sequence to generate a current vehicle damage information set includes: For each stitched vehicle body image in the stitched vehicle body image sequence, the following vehicle body damage identification steps are performed: The shallow image features are obtained by using the backbone image feature extraction network to extract shallow image features from the stitched vehicle image. In response to the first, second, or third side of the target vehicle corresponding to the stitched vehicle body image, deep image features are generated based on the deep feature extraction network of the first image and the shallow image features; In response to the fourth or fifth side of the target vehicle corresponding to the stitched vehicle body image, deep image features are generated based on the second image deep feature extraction network and the shallow image features, wherein the first image deep feature extraction network and the second image deep feature extraction network are set in parallel; The vehicle body skeleton features are generated by using the vehicle body skeleton feature extraction network and the shallow image features; Based on the damage locator and the deep image features, the region of interest for the damage is determined; Based on the damage type classifier and the local features of the region of interest in the deep image features, the damage type and damage confidence of the current vehicle body damage information are generated. The backbone image feature extraction network, the first deep image feature extraction network, the second deep image feature extraction network, the vehicle body skeleton feature extraction network, the damage locator, and the damage type classifier are included in the vehicle body damage recognition model. Feature shaping is performed on the local features of the region of interest in the damage in the deep image features to obtain the damage feature vector included in the current vehicle body damage information; The initial damage location is determined based on the region of interest. Based on the vehicle frame features and the initial damage location, the relative damage location is generated as part of the current vehicle damage information.

8. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 6.

9. A computer-readable medium, characterized in that, It stores a computer program thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images