Method, device, equipment and storage medium for determining vehicle placement posture

Abstract

Embodiments of the present application provide a method, apparatus, device, and storage medium for determining a vehicle's posture. The method comprises: acquiring an image to be detected, wherein the image to be detected includes at least a two-wheeled vehicle; performing scene segmentation on the image to be detected based on the category of the vehicle to obtain a segmentation result; and determining the vehicle's posture based on the segmentation result.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and are related to, but not limited to, a method, apparatus, device, and storage medium for determining a vehicle posture. Background Art

[0002] In urban environments, with the increasing number and brands of vehicles in urban areas, the orderly placement of vehicles on city roads has become a crucial aspect of urban management. However, the accuracy of the new intelligent urban vehicle tipping warning system in identifying vehicle placement is insufficient. Summary of the Invention

[0003] The embodiment of the present application provides a technical solution for determining the placement posture of a vehicle.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present invention provides a method for determining a vehicle posture, the method comprising:

[0006] Acquire an image to be detected, wherein the image to be detected includes at least a two-wheeled vehicle;

[0007] Based on the category of the vehicle, performing scene segmentation on the image to be detected to obtain a segmentation result;

[0008] Based on the segmentation result, the placement posture of the vehicle is determined.

[0009] In some embodiments, determining the vehicle's posture based on the segmentation result includes: determining, within the segmentation result, an image region containing two wheels of the vehicle; determining, within the image region, position information and a head orientation of the two wheels; and determining the posture of the two wheels based on the position information or the head orientation. Thus, determining the vehicle's position information and head orientation within the segmentation result allows for more accurate determination of the vehicle's posture.

[0010] In some embodiments, when the position information is the parking position of the two-wheeled vehicle, determining the placement posture of the two-wheeled vehicle based on the position information includes: determining an overlap between the parking position and a preset placement area in the image to be detected; and, if the parking position does not overlap with the preset placement area, determining that the placement posture of the two-wheeled vehicle is an irregular placement posture. In this way, by comparing whether the parking position of the two-wheeled vehicle overlaps with the preset placement area, vehicles with irregular placement postures can be screened out, thereby facilitating management personnel to manage irregularly placed vehicles.

[0011] In some embodiments, when the position information is the positional relationship between the two-wheeled vehicle and another object, determining the placement posture of the two-wheeled vehicle based on the position information includes: determining the contact area between the body of the two-wheeled vehicle and the other object; wherein the other object is an object other than the two-wheeled vehicle in the image to be detected; and determining that the placement posture of the two-wheeled vehicle is a tipped posture when the contact area is greater than a preset area threshold. In this manner, by determining the contact area between the body of the two-wheeled vehicle and the other object, a vehicle in a tipped posture can be accurately identified.

[0012] In some embodiments, when the other objects include the ground in the image to be detected and objects other than the two-wheeled vehicle, determining the contact area between the body of the two-wheeled vehicle and the other objects includes: determining a first contact area between the body of the two-wheeled vehicle and the ground; and, if the first contact area is less than a preset area threshold, determining a second contact area between the body of the two-wheeled vehicle and the object. In this manner, if the first contact area is smaller, further determining the contact area between the body of the two-wheeled vehicle and the object allows for more detailed identification of the two-wheeled vehicle in a tipping position.

[0013] In some embodiments, after determining the placement of the vehicle based on the segmentation results, the method further includes: if the placement is irregular or tilted, determining a first image region occupied by the two wheels of the vehicle; determining a ratio between the area of ​​the first image region and the area of ​​the image to be detected; and, in response to the ratio being greater than a preset ratio, outputting an alarm message to prompt adjustment of the placement of the two wheels. In this manner, outputting the alarm message can directly control the placement device to adjust the placement of the vehicle.

[0014] In some embodiments, after determining the ratio between the area of ​​the first image region and the area of ​​the image to be detected, the method further includes: in response to the ratio being less than a preset ratio, determining the number of the two-wheeled vehicles; and in response to the number being greater than a preset value, outputting the warning message. In this way, management personnel can be promptly alerted to effectively manage a large number of vehicles in irregular or overturned positions.

[0015] In some embodiments, after outputting an alarm message prompting adjustment of the two-wheeled vehicle's placement in response to the ratio being greater than a preset ratio, the method further includes: updating the alarm message based on the parking position of the two-wheeled vehicle, outputting the updated alarm message; and feeding the updated alarm message back to a placement device, causing the placement device to adjust the vehicle's placement based on the parking position of the two-wheeled vehicle. In this manner, by determining the vehicle's parking position, the placement device can be automatically controlled to adjust the vehicle's placement, thereby placing the vehicle in a suitable placement.

[0016] In some embodiments, the vehicle is at least one, and before determining the ratio between the area of ​​the first image area and the area of ​​the image to be detected, the method further includes: identifying a target vehicle including preset identification information in the at least one vehicle; and determining the ratio between the area of ​​the first image area and the area of ​​the image to be detected includes determining the ratio between the area of ​​a second image area occupied by the target vehicle and the area of ​​the image to be detected. In this way, targeted management of a specific type of vehicle can be achieved.

[0017] An embodiment of the present application provides a device for determining a vehicle posture, the device comprising:

[0018] A first acquisition module is configured to acquire an image to be detected, wherein the image to be detected includes at least a vehicle with two wheels;

[0019] A first segmentation module is used to perform scene segmentation on the image to be detected based on the category of the vehicle to obtain a segmentation result;

[0020] The first determination module is used to determine the placement posture of the vehicle based on the segmentation result.

[0021] Correspondingly, an embodiment of the present application provides a computer storage medium, on which computer executable instructions are stored. After the computer executable instructions are executed, the above-mentioned method for determining the vehicle placement posture can be implemented.

[0022] An embodiment of the present application provides a computer device, which includes a memory and a processor. The memory stores computer-executable instructions, and the processor can implement the above-mentioned method for determining the vehicle placement posture when running the computer-executable instructions on the memory.

[0023] Embodiments of the present application provide a method, apparatus, device, and storage medium for determining a vehicle's posture. For an image of a vehicle to be detected that includes at least two wheels, the method first performs scene segmentation on the image to be detected; then, based on the vehicle segmentation results, the vehicle's posture is determined. In this way, the vehicle's posture is identified based on effective segmentation of the vehicle in the image to be detected, enabling more accurate determination of the vehicle's posture. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of the implementation flow of the method for determining the vehicle placement posture provided in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of another implementation flow of the method for determining the vehicle placement posture provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of another implementation flow of the method for determining the vehicle placement posture provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of an application scenario of the method for determining the vehicle placement posture provided in an embodiment of the present application;

[0028] Figure 5 A semantic segmentation map for determining the vehicle posture of an image to be detected provided in an embodiment of the present application;

[0029] Figure 6 This is a schematic diagram of the structure of a device for determining a vehicle's posture according to an embodiment of the present application;

[0030] Figure 7 This is a schematic diagram of the composition structure of the computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the invention will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application but are not intended to limit the scope of the present application.

[0032] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0033] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0035] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0036] 1) Semantic segmentation: Separate various types of information in the image, such as marking the outline of a person in red and the road in purple. However, different people cannot be distinguished. This is equivalent to matching the external contours of the image with the labels of large categories.

[0037] 2) Residual Neural Network (ResNet), built from residual modules, uses a gradient descent algorithm to train a neural network. As the number of layers increases, the training error decreases. This approach can reach deeper layers of the network, helping to solve the problems of vanishing and exploding gradients, allowing for the training of deeper networks while maintaining good performance.

[0038] The following describes exemplary applications of the device for determining vehicle posture provided by embodiments of the present application. The device provided by embodiments of the present application can be implemented as various types of user terminals with image acquisition capabilities, such as laptop computers, tablet computers, desktop computers, cameras, mobile devices (e.g., personal digital assistants, dedicated messaging devices, portable gaming devices), and can also be implemented as a server. The following describes exemplary applications of the device when implemented as a terminal or server.

[0039] The method can be applied to a computer device, and the functions implemented by the method can be implemented by calling program codes by a processor in the computer device. Of course, the program codes can be stored in a computer storage medium. It can be seen that the computer device at least includes a processor and a storage medium.

[0040] The embodiment of the present application provides a method for determining the vehicle placement posture, such as Figure 1 As shown, combined with Figure 1 The following steps are explained:

[0041] Step S101: Acquire an image to be detected.

[0042] In some embodiments, the image to be detected includes at least a two-wheeled vehicle. The image to be detected can be a complex image including a non-motorized vehicle to be detected and a two-wheeled motor vehicle, or a simple image including a non-motorized vehicle to be detected and a two-wheeled motor vehicle. The image to be detected can be an image captured by any acquisition device. For example, if the non-motorized vehicle to be detected is a shared bicycle, the image captured in an urban area can include an image of the shared bicycle.

[0043] Step S102 : performing scene segmentation on the image to be detected based on the category of the vehicle to obtain a segmentation result.

[0044] In some embodiments, a semantic segmentation network can be used to perform scene segmentation on vehicles in the image to be detected to obtain a segmentation result. The semantic segmentation network is obtained by training a set of sample images in which different categories of vehicles are labeled in the picture. The semantic segmentation network is obtained by using a model with strong robustness to guide the training of a segmentation network with a simple structure to be trained, so that the semantic segmentation network has high performance while maintaining low complexity. The semantic segmentation network extracts features from the input image and performs semantic segmentation on the extracted features to generate a segmentation map that segments the different postures of the vehicles in the image.

[0045] Each sample image in the sample image set has been labeled with a preset semantic label set for different categories of vehicles in the picture. Taking vehicles in road traffic scenes in urban scenes as an example, the sample image set is images collected on the streets in urban scenes. The preset semantic labels in the preset semantic label set are set according to the requirements to be processed carried by the image to be detected. For example, the requirements to be processed carried by the image to be detected are vehicle monitoring requirements. In urban scenes, the requirements of the vehicle monitoring category include: supervising shared bicycles, electric vehicles, and parked cars, etc. The semantic label of the bicycle is set to 1, and then all bicycles that appear in the sample image are labeled as 1. In this way, the vehicles in the sample images are labeled with these preset semantic labels to obtain a sample image set that can be used as a training set to train the neural network to obtain a semantic segmentation network.

[0046] The semantic segmentation network includes: a residual neural network for feature extraction, a Deeplab v3 network for semantic segmentation of the extracted features, an upsampling module for upsampling the semantic segmentation results, and a final result prediction module.

[0047] In some possible implementations, the image to be detected is first fed into the residual neural network of a semantic segmentation network. This residual neural network then extracts features from the image to obtain a feature map, or image features. For example, an image of a shared bicycle is fed into the residual neural network of the semantic segmentation network to extract features from the image.

[0048] Then, the different placement postures of the two-wheeled vehicle are segmented in the image features to obtain the segmentation results.

[0049] In some embodiments, a semantic segmentation network to be trained is trained based on a preset semantic label set including different vehicle labels to obtain a trained semantic segmentation network. The trained semantic segmentation network is then used to segment the image to be detected within the image features, obtaining a segmentation result. The preset semantic label set includes labels representing various vehicle categories, as well as labels for other objects in the image and labels for the image background area, such as labels for various motor vehicles, labels for various non-motor vehicles, labels for other objects, and labels for the image background area. After feature extraction from the image to be detected using a residual neural network within the semantic segmentation network, the image features are input into a DeepLab v3 network to perform semantic segmentation of features corresponding to different vehicles within the image features. The preset semantic label set can also be set based on the number of wheels of the vehicle, such as labels for two-wheeled vehicles, three-wheeled vehicles, and four-wheeled vehicles. In some possible implementations, based on the preset semantic label set, a preset semantic label matching each vehicle category is determined within the image area occupied by the vehicles in the image to be detected. The segmentation result presents the preset semantic label matching each vehicle category. Taking vehicles parked on the roadside as an example, the preset semantic label set includes: label 0 for ordinary bicycles, label 1 for shared bicycles, label 2 for electric vehicles, label 3 for motorcycles, label 4 for other motor vehicles, and label 5 for the background area of ​​the image. If the parked vehicles in the image to be detected include: ordinary bicycles and shared bicycles, then the semantic segmentation network trained based on the preset semantic labels corresponding to ordinary bicycles and shared bicycles performs semantic segmentation on the image to be detected, and obtains a segmentation result that divides the image to be detected into three categories: ordinary bicycles, shared bicycles, and background areas.

[0050] Step S103: determining the placement posture of the vehicle based on the segmentation result.

[0051] In some embodiments, the vehicle includes a two-wheeled vehicle, such as an electric bicycle, an electric car, an ordinary bicycle, and a shared bicycle. The placement posture represents the posture of the parked vehicle, including: a dumped posture, a correct placement posture, a reverse placement posture, a chaotic placement posture, and a placement posture leaning against other objects. Since the segmentation results are obtained by segmenting different vehicles, there are preset semantic labels corresponding to each category in the segmentation results, so that the category of the vehicle can be determined based on the label. For example, the segmentation results include segmentation results of multiple different vehicles and other areas of the image; then, through the matching relationship between the preset semantic labels and the vehicles, the probability value corresponding to each type of vehicle in the image to be detected can be determined; thereby, the category with the largest probability value is determined as the final category of the vehicle in the image to be detected.

[0052] In an embodiment of the present application, for an image to be detected of a vehicle including at least two wheels, the image to be detected is first subjected to scene segmentation; and then the placement posture of the vehicle is determined based on the segmentation result of the vehicle; in this way, the placement posture of the vehicle can be determined more accurately based on the effective segmentation of the vehicle in the image to be detected.

[0053] In some embodiments, the vehicle's placement posture is determined by analyzing the segmentation results in combination with the vehicle's location information and the vehicle's head direction, that is, the above step S103 can be performed by Figure 2 The steps S201 to S203 shown implement:

[0054] Step S201: determining the image region where the two-wheeled vehicle is located in the segmentation result.

[0055] Here, since the segmentation result includes segmented vehicles of multiple categories and other image areas, the image area where the two-wheeled vehicle is located can be determined from the segmentation result; that is, the areas marked with different categories of vehicles in the segmentation result are the image areas where the two-wheeled vehicle is located.

[0056] Step S202: determining the position information of the two-wheeled vehicle and the front direction of the two-wheeled vehicle in the image area.

[0057] In some possible implementations, the location information of a two-wheeled vehicle includes: the parking position and the positional relationship between the vehicle and other objects; wherein the parking position can be understood as the world coordinates of the vehicle in the actual scene. Since the segmentation result includes each type of vehicle, the position of each type of vehicle in the image can be determined based on the matching relationship between the semantic label and the vehicle category; then, based on the position of the camera that captured the image to be detected and the correspondence between the camera coordinate system and the world coordinate system, the world coordinates of these vehicles, that is, the parking position in the actual scene, can be determined. For example, by analyzing the segmentation results, it is determined that the placement posture of the two-wheeled vehicle in the image to be detected is: a tilted posture; position regression is performed on the area of ​​the two-wheeled vehicle with the tilted posture to determine the position of the area of ​​the two-wheeled vehicle with the tilted posture in the image and the coordinate value of this position in the world coordinate system; thereby, the location information of the two-wheeled vehicle is obtained. By analyzing the relative positional relationship between the front of the two-wheeled vehicle and a fixed reference object in the image, where the fixed reference object is an object with a known orientation, the orientation of the front of the two-wheeled vehicle can be determined based on the relative positional relationship between the two and the orientation of the fixed reference object.

[0058] The positional relationship between the two-wheeled vehicle and other objects represents the leaning relationship between the two-wheeled vehicle and other fixed objects or movable objects in the image, for example, a two-wheeled vehicle leans on other vehicles.

[0059] Step S203: determining the placement posture of the two-wheeled vehicle based on the position information or the vehicle head orientation.

[0060] In some possible implementations, the two-wheeled vehicle's parking position is compared with a pre-set parking area, and the vehicle's head orientation is compared with the head orientation set in the pre-set parking area, thereby determining the specific placement of the two-wheeled vehicle. In this way, by determining the position information and head orientation of the two-wheeled vehicle in the segmentation result, the placement of the two-wheeled vehicle can be more accurately determined.

[0061] In some possible implementations, when the position information is the parking position of a two-wheeled vehicle, the following steps may be used to determine whether the two-wheeled vehicle is in an irregular placement posture, including:

[0062] The first step is to determine the overlapping relationship between the parking position and the preset placement area in the image to be detected.

[0063] Here, the preset placement area is a designated area for parking vehicles. After obtaining the parking position, it is determined whether the parking position of the two-wheeled vehicle is within the preset placement area. If the parking position of the two-wheeled vehicle is within the preset placement area, it is determined that the parking position overlaps with the preset placement area. Taking the two-wheeled vehicle as an example, the preset placement area can be an area designated for parking bicycles. If the bicycle's parking position is not within the preset placement area, it means that the parking position does not overlap with the preset placement area. If the bicycle's parking position is within the preset placement area, it means that the parking position overlaps with the preset placement area.

[0064] In the second step, when the parking position does not overlap with the preset placement area, determining that the placement posture of the two-wheeled vehicle is an irregular placement posture.

[0065] Here, the parking position does not overlap with the preset placement area, indicating that the two-wheeled vehicle is not parked within the preset placement area, i.e., the two-wheeled vehicle is not placed in accordance with the placement requirements. Therefore, the two-wheeled vehicle is determined to be placed in an irregular position. In this way, by comparing whether the parking position of the two-wheeled vehicle overlaps with the preset placement area, it is possible to screen out vehicles with irregular placement positions, thereby facilitating management personnel to manage irregularly placed vehicles.

[0066] In some possible implementations, when the position information is a positional relationship between a two-wheeled vehicle and other objects, the following steps may be used to determine whether the two-wheeled vehicle is in a tipping posture, including:

[0067] In a first step, a contact area between the body of the two-wheeled vehicle and the other object is determined.

[0068] Here, by performing image recognition on the image to be detected, the contact area between the body of the two-wheeled vehicle and other objects, as well as the area of ​​the body of the two-wheeled vehicle in the image to be detected, is determined. Other objects are objects other than the vehicle in the image to be detected. For example, the other objects are the ground on which the two-wheeled vehicle is parked in the image to be detected. In the image to be detected, the image area where the body of the two-wheeled vehicle contacts the ground is determined, and the area of ​​this image area is the contact area between the body of the two-wheeled vehicle and the ground.

[0069] In the second step, when the contact area is greater than a preset area threshold, determining that the placement posture of the two-wheeled vehicle is a dumping posture.

[0070] The preset area threshold can be the area of ​​the image region occupied by the vehicle body in the image to be detected, or it can be an area set according to a certain ratio based on the area of ​​the image region occupied by the vehicle body; for example, 0.5 times the area of ​​the image region occupied by the vehicle body is set as the preset area threshold. If the contact area is greater than the preset area threshold, it indicates that there is a large area of ​​contact between the two-wheeled vehicle and the other object. Since only part of the tires or the support of the two-wheeled vehicle are in contact with the parking ground when the vehicle is correctly positioned, there will not be a large contact area; therefore, if the contact area is greater than the preset area threshold, it indicates that the two-wheeled vehicle is in a tipping posture, which may be on the ground or on other objects. In this way, by judging the contact area between the vehicle body and other objects, a vehicle in a tipping posture can be accurately identified.

[0071] In some embodiments, when the other objects include objects other than the ground and the two-wheeled vehicle in the image to be detected, the contact area between the body of the two-wheeled vehicle and the other objects may be determined by the following steps, including:

[0072] In a first step, a first contact area between a body of the two-wheeled vehicle and the ground is determined.

[0073] By performing image recognition on the image to be detected, a contact area between the vehicle body and the ground, ie, a first contact area, is determined.

[0074] In the second step, when the first contact area is smaller than the preset area threshold, a second contact area between the body of the two-wheeled vehicle and the object is determined.

[0075] If the first contact area is large, it indicates that the two-wheeled vehicle may have fallen on the ground. If the first contact area is small, it indicates that the two-wheeled vehicle has not fallen on the ground. Further determination is made as to whether the vehicle has fallen onto another object to determine whether the two-wheeled vehicle is in a tipping position. Specifically, the image to be detected is recognized, and the area of ​​the image region where the body of the two-wheeled vehicle contacts the object is determined, i.e., the second contact area. If the second contact area is large, it indicates that the two-wheeled vehicle has fallen onto the object, and the two-wheeled vehicle is still determined to be in a tipping position. Thus, even if the first contact area is small, the second contact area between the body and the object is further determined, enabling more detailed identification of vehicles in a tipping position.

[0076] In some embodiments, after the placement posture of the vehicle to be detected is segmented in the image to be detected, different warning messages are output by analyzing the placement posture presented by the segmentation result to prompt the management personnel to perform corresponding management on the two-wheeled vehicle. That is, after step S103, the method further includes the following steps:

[0077] Step S141 : when the placement posture is an irregular placement posture or a tilted posture, determining a first image area occupied by two wheels of the vehicle.

[0078] In some embodiments, by analyzing the semantic labels of each category presented in the segmentation results, image regions occupied by vehicles of different categories are determined, i.e., the first image region. For example, by analyzing the semantic labels of two-wheeled vehicles included in the segmentation results, the image region occupied by the two-wheeled vehicles is determined, i.e., the first image region is obtained. If there are multiple image regions occupied by two-wheeled vehicles, these multiple image regions constitute the first image region.

[0079] Step S142: Determine the ratio between the area of ​​the first image region and the area of ​​the image to be detected.

[0080] In some embodiments, the area of ​​the first image region and the area of ​​the image to be detected are determined respectively, and the ratio between the two areas is determined; if the first image region includes two image regions, then the area of ​​the first image region is the sum of the areas of the two image regions.

[0081] In some possible implementations, when there is at least one vehicle, first, a target vehicle including preset identification information is determined in at least one vehicle; wherein the preset identification information is information that can indicate the category of the device, such as the serial number of the device or the brand of the device, etc. In a specific example, the preset identification information is a shared bicycle, that is, in at least one vehicle, the target vehicle is determined to be a shared bicycle. Then, the ratio between the area of ​​the second image area occupied by the target vehicle and the area of ​​the image to be detected is determined. For example, multiple vehicles include shared bicycles, electric vehicles, and ordinary bicycles. In the image to be detected, the ratio between the area of ​​the second image area occupied by the shared bicycle among the multiple vehicles and the area of ​​the entire image is determined. In this way, by determining the area ratio between the image area occupied by a certain category of vehicles and the entire image area, targeted management of a certain category of vehicles can be achieved.

[0082] Step S143: In response to the ratio being greater than a preset ratio, outputting an alarm message to prompt the user to adjust the placement of the two-wheeled vehicle.

[0083] In some embodiments, taking the two-wheeled vehicle as a non-motor vehicle as an example, the preset ratio can be determined based on the category of the preset posture and the pending processing requirements. For example, if the category of the preset posture is a tipping posture and the pending processing requirement is to lift up the non-motor vehicle in the tipping posture, the preset ratio is set to be greater than or equal to 20%. If the category of the preset posture is a correct placement posture and the pending processing requirement is to monitor the placement position of the non-motor vehicle in the correct placement posture, the preset ratio is set to be greater than or equal to 10% and less than or equal to 50%. Taking the preset posture as a tipping posture as an example, if the ratio between the area of ​​the first image area and the area of ​​the image to be detected is greater than or equal to 30%, then an alarm message is sent to the management end to prompt the management end to manage multiple target non-motor vehicles in a tipping posture. Taking the non-motor vehicle as a shared bicycle and the preset posture as a tipping posture as an example, if in the image to be detected, the image area occupied by the shared bicycle to be detected in the tipping posture is more than 30% of the entire image area, then an alarm message is generated and sent to the management end to prompt the management end to manage these tipped shared bicycles.

[0084] The above steps S141 to S143 provide a method of outputting corresponding alarm information by comparing the area of ​​the first image region occupied by multiple two-wheeled vehicles with irregular placement postures or tilted postures with the area of ​​the entire image. The alarm information can be output to the management end, or it can directly control the placement equipment to adjust the posture of the two-wheeled vehicle.

[0085] Step S144 , in response to the ratio being less than the preset ratio, determining the number of the two-wheeled vehicles.

[0086] In some embodiments, if the ratio between the area of ​​the first image region and the area of ​​the image to be detected is small. For example, if the vehicle is a non-motor vehicle, and the ratio between the area of ​​the first image region and the area of ​​the image to be detected is less than 30%, the number of non-motor vehicles in a tipping position is determined. In a specific example, if the non-motor vehicle is a shared bicycle, and the ratio of the image area occupied by the shared bicycle in a tipping position to the area of ​​the entire image is less than 30%, the number of shared bicycles in a tipping position is determined.

[0087] Step S145: In response to the number being greater than a preset value, output the alarm information.

[0088] In some embodiments, if a large number of vehicles are in a preset posture, an alert message is still output. In a specific example, if the vehicles are shared bicycles, the ratio of the image area occupied by shared bicycles in a tipping posture to the total image area is less than 30%, and the number of these shared bicycles in a tipping posture is greater than 20, then an alert message is output to the management terminal to inform the management that a large number of shared bicycles are in a tipping posture.

[0089] In an embodiment of the present application, by marking preset semantic labels on pixels in the segmentation results, the image area occupied by two-wheeled vehicles with irregular or fallen postures is determined. If the area of ​​the image area is large, an alarm information is output to the management end to promptly remind the management personnel to effectively manage the vehicles with irregular or fallen postures.

[0090] In some embodiments, by analyzing the position of the vehicle in the image to be detected, the position information of the two-wheeled vehicle in the actual scene can be determined, and then the placement device can be controlled to adjust the placement posture of the two-wheeled vehicle. This can be achieved by the following steps:

[0091] In a first step, the warning information is updated based on the parking position of the two-wheeled vehicle, and the updated warning information is output.

[0092] In some embodiments, the parking position of the two-wheeled vehicle represents the world coordinates of the vehicle in the actual scene. The parking position is carried in the warning information, thereby updating the warning information and obtaining and outputting the updated warning information.

[0093] The second step is to feed back the updated warning information to the placement device so that the placement device adjusts the placement posture of the vehicle based on the parking position of the two-wheeled vehicle.

[0094] In some embodiments, the placement device is any type of device that can adjust the placement posture of the two-wheeled vehicle, such as a robot or a robotic arm that adjusts the placement posture of the two-wheeled vehicle.

[0095] In other embodiments, updated alarm information can be output to a management terminal, while simultaneously controlling the placement device to adjust the placement of the two-wheeled vehicle based on the location information. For example, in the case of a shared bicycle, the parking location of a toppled shared bicycle can be determined, and the robot can be controlled to automatically move to that location and lift the toppled shared bicycle. In this way, by determining the parking location of the vehicle, the placement device can be automatically controlled to adjust the vehicle's placement, thereby ensuring that the vehicle is in the appropriate placement.

[0096] Below, we will illustrate the exemplary application of an embodiment of the present application in an actual application scenario, taking the created label library and the vehicles as bicycles of different categories to illustrate the segmentation and positioning of bicycle dumping areas in urban road scenarios.

[0097] With the increasing number of shared bicycle brands and numbers in urban areas, the orderly placement of shared bicycles on urban roads has become a crucial aspect of urban management. This issue primarily focuses on whether shared bicycles are dumped. Traditionally, bicycle dumping warning systems rely on manual labor, resulting in high enforcement costs and low efficiency. New intelligent urban bicycle dumping warning systems can significantly reduce labor costs and enable efficient, 24 / 7 management. However, accurate segmentation and localization of bicycle dumping areas on urban roads is fundamental to these warnings. Determining the posture of bicycles is a key issue in computer vision and intelligent video surveillance, aiming to locate both semantic and instance elements within a scene.

[0098] However, in practical applications, segmentation modeling is complex due to factors such as lighting conditions, background clutter, and changing viewpoints. In related technologies, the lack of in-depth research and analysis of bicycle ordering in urban settings has led to a lack of consensus on model definitions and a practically usable labeling system.

[0099] Based on this, the embodiment of the present application provides a method for determining the placement of vehicles. By creating a unified and standardized semantic tag library for bicycle order in urban scenes, the semantic tag library can comprehensively cover urban bicycle order scenes and meet actual needs. In addition, the semantic tag library system is independent, complete, and basic. Based on the above semantic tag library, data definition and annotation specifications are constructed, thereby forming a lightweight semantic segmentation network. Based on the segmentation results, the regional position of each bicycle in the scene is determined, and then the placement of the bicycle is analyzed. The implementation process is as follows:

[0100] The first step is to define the semantic labels of bicycle categories and the corresponding segmentation results in urban scenes according to actual needs (corresponding to the needs to be processed in the above embodiment), create semantic labels of bicycle categories, and obtain a semantic label library.

[0101] In some embodiments, due to the irregular shape of the bicycles, a pile of bicycles may occur. The bicycles are labeled according to their outlines. When a bicycle is blocked, it can be labeled by imaginary labeling. To increase the labeling speed, polygons are used to label the bicycles in the embodiment of this application. The semantic labels for the bicycle category and the corresponding segmentation results include:

[0102] a) The category id is 0, the semantic label is shared bicycle, and the segmentation result refers to shared bicycle.

[0103] b) The category ID is 1, the semantic label is the ordinary bicycle area, and the corresponding segmentation result refers to other bicycle areas that are not shared bicycles, including ordinary bicycles in motion, etc.

[0104] The second step is to train the semantic segmentation network to be trained based on the semantic tag library to obtain a trained semantic segmentation network.

[0105] The third step is to use the trained semantic segmentation network to perform semantic segmentation on the input image to be detected to obtain a semantic segmentation map.

[0106] In some possible implementations, the segmentation result can be formatted as a two-dimensional matrix of the same size as the original image, representing a semantic segmentation map. The semantic segmentation map includes three numbers: 0, 1, and 2, representing the background, the normal motor vehicle area, and the shared bicycle, respectively.

[0107] The fourth step is to determine the location of the area where each type of bicycle is located in the semantic segmentation map.

[0108] The method for determining the vehicle placement posture provided in the embodiment of the present application can be performed by Figure 3 The steps shown are implemented, Figure 3 Another implementation flow diagram of vehicle category segmentation provided in the embodiment of the present application, combined with Figure 3 The steps shown are explained below:

[0109] Step S301: Acquire an image to be detected including a bicycle image.

[0110] Step S302: input the image into the residual neural network for feature extraction.

[0111] Step S303: Input the extracted features into the convolutional network model to perform semantic segmentation to obtain a semantic segmentation result.

[0112] In some possible implementations, the convolutional network model may be an Atrous Spatial Pyramid Pooling (ASPP) model with dilated convolutions, which may be implemented using a Deeplab v3 network.

[0113] Step S304: upsample the semantic segmentation result to obtain an upsampled result.

[0114] Step S305: Based on the upsampling result, the location areas of bicycles of different categories are output.

[0115] In some possible implementations, after upsampling the semantic segmentation results, i.e., amplifying the semantic segmentation results, the final detection results corresponding to the semantic segmentation results can be obtained based on the matching relationship between the semantic labels and the segmentation results; based on the category of the bicycle indicated in the detection result, the parking position and the front direction of the shared bicycle are analyzed, and the placement posture of the shared bicycles in different categories can be determined. Figure 4 As shown, Figure 4 Schematic diagram of an application scenario of the method for determining the parking posture of a vehicle provided in an embodiment of the present application, wherein image 401 is a captured image to be detected, including: a shared bicycle 411 in a tipped posture, a shared bicycle 412 in an untipped posture, and an ordinary bicycle 413. Image 401 is input into a residual neural network for feature extraction, and then the feature extraction result is input into the Deeplab v3 network to perform semantic segmentation on the different bicycles in image 401 to obtain a semantic segmentation result; based on the segmentation result, the parking postures of different shared bicycles are determined, see Figure 5 As shown, Figure 5 The embodiment of the present application provides a semantic segmentation map for determining the parking posture of the image to be detected; that is, the analysis results of the parking postures of different shared bicycles, as shown in image 501, where area 511 represents a shared bicycle 411 in a tipped-over posture, area 512 represents a shared bicycle 412 in an untipped posture, and area 513 represents an ordinary bicycle 413. In this way, by standardizing the shared bicycle label definition specifications, data specifications and unified interfaces for urban scenes, it can be ensured that the created semantic label library clearly covers bicycle scenes, is practical and easy to operate; and the positioning network used can quickly segment the shared bicycles that are tipped over and not tipped over for external warning.

[0116] In an embodiment of the present application, by comprehensively analyzing a large number of bicycle scenes on urban roads, a complete, independent and basic semantic segmentation label system is defined, thereby providing a lightweight semantic segmentation algorithm framework that is easy and quick to use, while outputting standard semantic segmentation results.

[0117] The embodiment of the present application provides a device for determining a vehicle's posture. Figure 6 This is a schematic diagram of the structure of the vehicle posture determination device according to an embodiment of the present application. Figure 6 As shown, the vehicle posture determination device 600 includes:

[0118] A first acquisition module 601 is configured to acquire an image to be detected, wherein the image to be detected includes at least a two-wheeled vehicle;

[0119] A first segmentation module 602 is configured to perform scene segmentation on the image to be detected based on the category of the vehicle to obtain a segmentation result;

[0120] The first determination module 603 is configured to determine the placement posture of the vehicle based on the segmentation result.

[0121] In some embodiments, the first determining module 603 includes:

[0122] A first determining submodule is configured to determine, in the segmentation result, an image region where two wheels of the vehicle are located;

[0123] A second determining submodule is configured to determine, in the image area, position information of the two-wheeled vehicle and a frontal orientation of the two-wheeled vehicle;

[0124] The third determining submodule is used to determine the placement posture of the vehicle based on the position information or the vehicle head orientation.

[0125] In some embodiments, when the position information is the parking position of the two-wheeled vehicle, the third determining submodule includes:

[0126] A first determining unit is configured to determine an overlapping relationship between the parking position and a preset placement area in the image to be detected;

[0127] The second determining unit is configured to determine that the placement posture of the two-wheeled vehicle is an irregular placement posture when the parking position does not overlap with the preset placement area.

[0128] In some embodiments, when the position information is the positional relationship between the two-wheeled vehicle and another object, the third determining submodule includes:

[0129] a third determining unit, configured to determine a contact area between the body of the two-wheeled vehicle and the other object; wherein the other object is an object other than the two-wheeled vehicle in the image to be detected;

[0130] The fourth determining unit is configured to determine that the placement posture of the two-wheeled vehicle is a dumping posture when the contact area is greater than a preset area threshold.

[0131] In some embodiments, when the other objects include objects other than the ground and the two-wheeled vehicle in the image to be detected, the third determining unit includes:

[0132] a first determining subunit, configured to determine a first contact area between a body of the two-wheeled vehicle and the ground;

[0133] The second determining subunit is configured to determine a second contact area between the body of the two-wheeled vehicle and the object when the first contact area is smaller than the preset area threshold.

[0134] In some embodiments, the apparatus further comprises:

[0135] a second determining module, configured to determine a first image area occupied by the two-wheeled vehicle when the placement posture is an irregular placement posture or a tilted posture;

[0136] a third determining module, configured to determine a ratio between an area of ​​the first image region and an area of ​​the image to be detected;

[0137] The first output module is configured to output an alarm message in response to the ratio being greater than a preset ratio, so as to prompt adjustment of the placement posture of the two-wheeled vehicle.

[0138] In some embodiments, the apparatus further comprises:

[0139] a fourth determining module, configured to determine the number of the two-wheeled vehicles in response to the ratio being less than the preset ratio;

[0140] The second output module is configured to output the warning information in response to the number being greater than a preset value.

[0141] In some embodiments, the apparatus further comprises:

[0142] a third output module, configured to update the warning information based on the parking position of the two-wheeled vehicle and output the updated warning information;

[0143] The first feedback module is used to feed back the updated warning information to the placement device, so that the placement device adjusts the placement posture of the two-wheeled vehicle based on the parking position of the two-wheeled vehicle.

[0144] In some embodiments, the vehicle is at least one, and the apparatus further comprises: a fifth determining module configured to determine, among the at least one vehicle, a target vehicle including preset identification information;

[0145] The third determination module is further used to determine the ratio between the area of ​​the second image region occupied by the target vehicle and the area of ​​the image to be detected.

[0146] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0147] It should be noted that, in the embodiment of the present application, if the above-mentioned method for determining the vehicle placement posture is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a terminal, server, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a sports hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0148] Correspondingly, an embodiment of the present application further provides a computer program product, which includes computer-executable instructions. After the computer-executable instructions are executed, the method for determining the vehicle placement posture provided in the embodiment of the present application can be implemented.

[0149] An embodiment of the present application further provides a computer storage medium having computer executable instructions stored thereon. When the computer executable instructions are executed by a processor, the method for determining the vehicle placement posture provided in the above embodiment is implemented.

[0150] The embodiment of the present application provides a computer device, Figure 7 This is a schematic diagram of the structure of the computer device according to the embodiment of the present application. Figure 7 As shown, the computer device 700 includes: a processor 701, at least one communication bus, a communication interface 702, at least one external communication interface, and memory 703. Communication interface 702 is configured to enable communication between these components. Communication interface 702 may include a display screen, and the external communication interface may include a standard wired interface or a wireless interface. Processor 701 is configured to execute an image processing program in memory to implement the vehicle posture determination method provided in the above-described embodiment.

[0151] The descriptions of the above embodiments of the vehicle posture determination device, computer device, and storage medium are similar to those of the above-mentioned method embodiments, and have similar technical descriptions and beneficial effects as the corresponding method embodiments. Due to space limitations, they can be referred to the description of the above-mentioned method embodiments and are not further described here. For any technical details not disclosed in the embodiments of the vehicle posture determination device, computer device, and storage medium of this application, please refer to the description of the method embodiments of this application for an understanding.

[0152] It should be understood that references to "one embodiment" or "an embodiment" throughout this specification mean that specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the appearance of "in one embodiment" or "in an embodiment" throughout this specification does not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of the present application, the order of execution of the above-mentioned processes does not necessarily indicate the order in which they are executed. The order of execution of each process should be determined by its function and inherent logic and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. It should be noted that, in this document, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising that element.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0154] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0155] In addition, the functional units in the embodiments of the present application can all be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units. It can be understood by ordinary technicians in this field that all or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions, and the above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiments; and the above-mentioned storage medium includes: various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), magnetic disks or optical disks.

[0156] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks. The above is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for determining the placement posture of a two-wheeled vehicle, characterized in that: The method comprises: Acquire an image to be detected, wherein the image to be detected includes at least two-wheeled vehicles of different categories; the two-wheeled vehicles of different categories include at least one of the following: ordinary bicycles, shared bicycles, electric vehicles, motorcycles, and other motor vehicles; Features of the two-wheeled vehicle in the image to be detected are extracted using a semantic segmentation network, and semantic segmentation is performed on the extracted features using the semantic segmentation network to generate a segmentation result; wherein the segmentation result is used to present preset semantic labels that match the two-wheeled vehicle in each category; the semantic segmentation network is obtained by training a semantic segmentation network to be trained based on a preset semantic label set including different vehicle labels; the preset semantic label set includes: labels representing the two-wheeled vehicles of the different categories, labels of other objects in the image, and labels of the image background area; wherein the other objects are objects other than the two-wheeled vehicle in the image to be detected; In the segmentation result, determining an image region where two wheels of the vehicle are located; determining, in the image area, a positional relationship between the two-wheeled vehicle and the other objects; Determining a placement posture of the two-wheeled vehicle based on a positional relationship between the two-wheeled vehicle and the other objects, so that the two-wheeled vehicle can be parked in a standardized manner based on a placement device; the placement device is a robot or a mechanical arm capable of adjusting the placement posture of the two-wheeled vehicle; Wherein, determining the placement posture of the two-wheeled vehicle based on the positional relationship between the two-wheeled vehicle and the other object includes: In a case where the other objects include the ground in the image to be detected and objects other than the two-wheeled vehicle, determining a first contact area between a body of the two-wheeled vehicle and the ground; determining a second contact area between the body of the two-wheeled vehicle and the object when the first contact area is less than a preset area threshold; When the second contact area is greater than a preset area threshold, it is determined that the placement posture of the two-wheeled vehicle is a dumping posture.

2. The method according to claim 1, characterized in that After determining the image region where the two-wheeled vehicle is located in the segmentation result, the method further includes: determining, in the image area, a parking position of the two-wheeled vehicle; Determining an overlapping relationship between the parking position and a preset placement area in the image to be detected; In a case where the parking position does not overlap with the preset placement area, the placement posture of the two-wheeled vehicle is determined to be an irregular placement posture.

3. The method according to claim 1 or 2, characterized in that After determining the placement posture of the vehicle based on the segmentation result, the method further includes: In a case where the placement posture is an irregular placement posture or a tilted posture, determining a first image area occupied by two wheels of the vehicle; determining a ratio between an area of ​​the first image region and an area of ​​the image to be detected; In response to the ratio being greater than a preset ratio, an alarm message is output to prompt the vehicle to adjust the placement posture of the two-wheeled vehicle.

4. The method according to claim 3, characterized in that After determining the ratio between the area of ​​the first image region and the area of ​​the image to be detected, the method further includes: In response to the ratio being less than the predetermined ratio, determining the number of the two-wheeled vehicles; In response to the number being greater than a preset value, the warning information is output.

5. The method according to claim 3, characterized in that After outputting an alarm message in response to the ratio being greater than a preset ratio to prompt the user to adjust the placement of the two-wheeled vehicle, the method further includes: updating the warning information based on the parking position of the two-wheeled vehicle and outputting the updated warning information; The updated warning information is fed back to the placement device so that the placement device adjusts the placement posture of the two-wheeled vehicle based on the parking position of the two-wheeled vehicle.

6. The method according to claim 3, characterized in that The vehicle is at least one, and before determining the ratio between the area of ​​the first image region and the area of ​​the image to be detected, the method further includes: determining a target vehicle including preset identification information among the at least one vehicle; Determining the ratio between the area of ​​the first image region and the area of ​​the image to be detected includes: determining the ratio between the area of ​​the second image region occupied by the target vehicle and the area of ​​the image to be detected.

7. A device for determining the placement posture of a two-wheeled vehicle, characterized in that: The device comprises: A first acquisition module is configured to acquire an image to be detected, wherein the image to be detected includes at least two-wheeled vehicles of different categories; the two-wheeled vehicles of different categories include at least one of the following: ordinary bicycles, shared bicycles, electric vehicles, motorcycles, and other motor vehicles; A first segmentation module is configured to extract features of the two-wheeled vehicle in the image to be detected using a semantic segmentation network, and perform semantic segmentation on the extracted features using the semantic segmentation network to generate a segmentation result; wherein the segmentation result is used to present preset semantic labels that match the two-wheeled vehicle in each category; the semantic segmentation network is obtained by training a semantic segmentation network to be trained based on a preset semantic label set including different vehicle labels; the preset semantic label set includes: labels representing the two-wheeled vehicles of the different categories, labels of other objects in the image, and labels of the image background area; wherein the other objects are objects other than the two-wheeled vehicle in the image to be detected; a first determination module configured to determine, from the segmentation result, an image region in which a two-wheeled vehicle among the vehicles is located; determine, within the image region, a positional relationship between the two-wheeled vehicle and the other objects; and determine, based on the positional relationship between the two-wheeled vehicle and the other objects, a placement posture of the two-wheeled vehicle, so that the two-wheeled vehicle can be parked in a standardized manner using a placement device; the placement device being a robot or a robotic arm capable of adjusting the placement posture of the two-wheeled vehicle; The first determination module is further configured to determine a first contact area between the body of the two-wheeled vehicle and the ground when the other objects include the ground in the image to be detected and objects other than the two-wheeled vehicle; determine a second contact area between the body of the two-wheeled vehicle and the object when the first contact area is less than a preset area threshold; and determine that the placement posture of the two-wheeled vehicle is a tilted posture when the second contact area is greater than a preset area threshold.

8. A computer storage medium, characterized in that The computer storage medium stores computer executable instructions, which, after being executed, can implement the method for determining the vehicle placement posture according to any one of claims 1 to 6.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores computer-executable instructions, and the processor can implement the method for determining the vehicle placement posture according to any one of claims 1 to 6 when running the computer-executable instructions on the memory.

Images