Method and device for maintaining ground identifier, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a ground identifier maintenance method and device, equipment, a storage medium and a program product. The method comprises the steps of obtaining a target image associated with a vehicle returning area in response to a vehicle returning request for a target vehicle; based on the target image, determining the damage degree of a ground identifier associated with the vehicle returning area; and in response to the condition that the damage degree of the ground identifier is greater than a threshold value, sending a notification associated with maintenance of the ground identifier. According to the embodiment of the invention, the efficiency of maintaining the ground identifier can be improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and particularly to methods, apparatuses, devices, storage media, and computer presentation products for maintaining ground markings. Background Art

[0002] Shared vehicles have become an important mode of transportation. Taking two-wheeled vehicles such as shared bicycles as an example, users can conveniently control the vehicles with their portable devices (e.g., mobile phones) to complete various operations such as unlocking, locking, paying, and account management. When a user finishes using a shared bicycle, the shared bicycle needs to be returned. The ground markings at the return location can guide the user to the designated location to return the shared bicycle. Summary of the Invention

[0003] In a first aspect of the present disclosure, a method for maintaining ground markings is provided. The method includes: in response to a return request for a target vehicle, obtaining a target image associated with the return area; based on the target image, determining the degree of damage to the ground markings associated with the return area; and in response to the degree of damage to the ground markings being greater than a threshold, sending a notice associated with maintaining the ground markings.

[0004] In a second aspect of the present disclosure, an apparatus for maintaining ground markings is provided. The apparatus includes: an obtaining module configured to obtain a target image associated with the return area in response to a return request for a target vehicle; a determining module configured to determine the degree of damage to the ground markings associated with the return area based on the target image; and a sending module configured to send a notice associated with maintaining the ground markings in response to the degree of damage to the ground markings being greater than a threshold.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to execute the method of the first aspect when executed by the at least one processing unit.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided. A computer program is stored on the computer-readable storage medium, and the computer program is executable by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided. The computer program product includes computer-executable instructions, where the computer-executable instructions implement the method of the first aspect when executed by a processor.

[0008] It should be understood that the content described in this section is not intended to define the key features or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0010] Figure 1 FIG. shows a schematic diagram of an exemplary environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 FIG. shows a flowchart of an exemplary process for maintaining ground markings according to some embodiments of the present disclosure;

[0012] Figures 3A to 3D FIG. shows a schematic diagram of detecting ground markings according to some embodiments of the present disclosure;

[0013] Figure 4 FIG. shows a flowchart of an exemplary process for maintaining ground markings according to some embodiments of the present disclosure;

[0014] Figure 5 FIG. shows a schematic structural block diagram of an exemplary apparatus for maintaining ground markings according to some embodiments of the present disclosure; and

[0015] Figure 6 FIG. shows a block diagram of an electronic device capable of implementing multiple embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0017] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this document, and any type of embodiment can be included under any section / subsection. Additionally, the embodiments described in any section / subsection can be combined with any other embodiments described in the same section / subsection and / or different sections / subsections in any manner.

[0018] In the description of the embodiments of the present disclosure, the term "including" and its similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". There may also be other explicit and implicit definitions hereinafter. The terms "first", "second", etc. may refer to different or the same objects. There may also be other explicit and implicit definitions hereinafter.

[0019] As used herein, a "unit", "operation unit" or "sub-unit" may consist of a machine learning model or network of any suitable structure. As used herein, a set of elements or similar expressions may include one or more such elements. For example, "a set of convolutional units" may include one or more convolutional units.

[0020] The embodiments of the present disclosure may involve the user's data, data acquisition and / or use, etc. These aspects all comply with the corresponding laws, regulations and related provisions. In the embodiments of the present disclosure, the collection, acquisition, processing, processing, forwarding, use, etc. of all data are carried out on the premise that the user is aware and confirms. Accordingly, when implementing the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained through appropriate means in accordance with the relevant laws and regulations. The specific notification and / or authorization methods may vary according to the actual situation and application scenarios, and the scope of the present disclosure is not limited in this regard.

[0021] As briefly discussed above, shared vehicles have become an important mode of transportation. The service platform of shared vehicles needs to assist users in performing operations such as fixed-point vehicle return, in-pen vehicle return, and orderly vehicle return. When a user finishes using a shared bicycle, the shared bicycle needs to be returned. The ground markings at the vehicle return location can guide the user to return the shared bicycle to the designated location. The user relies on the guidance of the ground markings when looking for the vehicle return location. When the ground markings are damaged, it will affect the vehicle return efficiency and accuracy of the user.

[0022] The embodiments of the present disclosure propose a solution for maintaining ground markings. According to this solution: in response to a vehicle return request for a target vehicle, obtain a target image associated with the vehicle return area; based on the target image, determine the damage degree of the ground markings associated with the vehicle return area; and in response to the damage degree of the ground markings being greater than a threshold, send a notice associated with maintaining the ground markings.

[0023] In this way, embodiments of the present disclosure can determine the damage degree of the ground markings associated with the return area by detecting the target image of the return area corresponding to the vehicle return request. Further, embodiments of the present disclosure can send a notice associated with maintaining the ground markings in response to the damage degree of the ground markings being greater than a threshold. In this way, embodiments of the present disclosure can detect the damage degree of the ground markings during the vehicle return process of the user, improve the efficiency of maintaining the ground markings, thus meeting the vehicle return needs of the user and improving the vehicle return experience of the user.

[0024] Various example implementations of the solution will be further described in detail below with reference to the accompanying drawings.

[0025] Example environment

[0026] Figure 1 FIG. 100 is a schematic diagram showing an example environment 100 in which embodiments of the present disclosure can be implemented. The environment 100 includes a target vehicle 130, a control terminal 110, a user 140, a user device 120 used by the user 140, and an image acquisition device 135 configured in the target vehicle.

[0027] In some embodiments, an application program can be installed and run on the user device 120, and an interface for the user 140 to operate can be presented. The user device 120 can be a terminal device installed and running a shared vehicle application, including but not limited to a personal computer, a smart phone, a smart watch, smart glasses, a smart helmet, a laptop computer, a tablet computer, a personal digital assistant, or any other suitable portable electronic device.

[0028] In some embodiments, the target vehicle 130 is a shared vehicle, for example, a shared two-wheeler, a shared electric two-wheeler, a shared car, or any other shared transportation means that complies with laws and regulations. Further, the target vehicle 130 can include a central control unit, which can be configured to manage and control the functions of the target vehicle 130. In some embodiments, according to needs, the central control unit can communicate with the control terminal 110 and the user device 120 wirelessly respectively.

[0029] In some embodiments, the target vehicle 130 can be configured with an image acquisition device 135. The image acquisition device 135 can be implemented as an image sensor (for example, a camera, etc.) or other sensors that can obtain parking information. As an example, the image acquisition device 135 can obtain image data associated with the parking of the target vehicle. Such image data can include, for example, image content associated with the ground markings in the return area.

[0030] In some implementations, the control terminal 110 may be a remote server. In this case, the control terminal 110 may be configured as a remote server for centrally managing and controlling various shared vehicles including the target vehicle 130. For example, the control terminal 110 may manage and control the target vehicle 130 at least by communicating with the central control unit on the target vehicle 130.

[0031] In some embodiments, the control terminal 110 may be implemented by an independent server or a server cluster composed of multiple servers. In other words, the control terminal 110 may be one or a group of servers providing service functions, or one or a group of virtual machines that can provide services. The present disclosure is not limited in terms of the implementation form of the control terminal 110.

[0032] In some embodiments, at least some functions of the control terminal 110 may be implemented by the user device 120.

[0033] As Figure 1 shown, the control terminal 110 may include a service platform 112 and an algorithm platform 114. In some embodiments, the service platform 112 may manage the service process between the target vehicle 130 and the user device 120, and manage and maintain the data generated during the shared bicycle service. The algorithm platform 114 may be used to implement operations with relatively high computational complexity such as image processing and logical reasoning. In operation, the service platform 112 may call the interface of the algorithm platform 114 and send the data to be processed to the algorithm platform 114. Further, the service platform 112 may receive the processing results returned by the algorithm platform 114 from the algorithm platform 114 to better implement the service of the shared vehicle.

[0034] It should be understood that the structure and functions of the environment 100 are described only for exemplary purposes, without implying any limitation on the scope of the present disclosure.

[0035] Example request processing

[0036] Figure 2 shows a flowchart of an example process 200 for maintaining ground markings according to some embodiments of the present disclosure. The process 200 may be implemented at the control terminal 110, the user device 120, and / or the target vehicle 130. For ease of discussion, reference Figure 1 is made to the environment 100 for discussion.

[0037] In block 205, the control terminal 110 and / or the user device 120 may receive the user's vehicle return request.

[0038] As an example, after user 140 finishes using the vehicle, user 140 sends a vehicle return request for target vehicle 130 to control terminal 110. For example, user 140 may send the vehicle return request through user device 120. For example, when user 140 triggers a control for requesting vehicle return or trip end in the user interface presented by the shared vehicle application of user device 120. Based on this trigger operation, user device 120 sends a vehicle return request for target vehicle 130 to control terminal 110.

[0039] As an example, user 140 may also send a vehicle return request to control terminal 110 based on the electronic device configured on target vehicle 130.

[0040] In some embodiments, control terminal 110 may, in response to the vehicle return request for target vehicle 130, obtain a target image associated with the vehicle return area.

[0041] As an example, at block 210, control terminal 110 may identify whether the target vehicle enters the pen based on the camera. For example, control terminal 110 may obtain a target image associated with target vehicle 130.

[0042] As an example, control terminal 110 may use the image acquisition device 135 (such as a camera or other image sensor, etc.) deployed on target vehicle 130 to obtain a target image associated with the vehicle return area. In this way, the embodiments of the present disclosure can directly obtain the target image through the image acquisition device deployed on target vehicle 130, saving labor costs and improving the acquisition efficiency.

[0043] In some embodiments, image acquisition device 135 may be configured at a preset position of target vehicle 130. For example, the preset position may include the front position of target vehicle 130 (for example, the position near the front wheel of the shared vehicle) and / or the rear position (for example, the position near the rear wheel of the shared vehicle). For example, since the standard parking of the shared vehicle is generally that the front position or the rear position is aligned with the ground marking, installing image acquisition device 135 at the front position and / or the rear position can facilitate obtaining the target image associated with the ground marking. As an example, the number of image acquisition devices 135 may be one or more (for example, one image acquisition device 135 is installed at each of the front position and the rear position).

[0044] At block 215, control terminal 110 may determine whether target vehicle 130 enters the pen based on the target image.

[0045] As an example, control terminal 110 may process the target image by using a preset machine learning model to identify whether the ground marking and / or target vehicle 130 is included in the target image.

[0046] As an example, the ground marking may include a parking line marking associated with a preset color. Exemplarily, the parking line marking associated with the preset color may include, for example, a single-color parking line, a blue-and-white double line, a red-and-white double line, and / or a cyan-and-white double line, etc.

[0047] As an example, the ground marking may include a parking line marking associated with a preset shape. Exemplarily, the parking line marking associated with the preset shape may include, for example, a green T-line, etc.

[0048] Further, the control terminal 110 may determine whether the target vehicle 130 enters the pen (i.e., whether the shared vehicle 130 is inside the return area) based on the relative position of the ground marking and the target vehicle 130 (for example, the parking orientation of the target vehicle 130 relative to the ground marking). The present disclosure does not aim to limit the specific determination process of whether the target vehicle 130 enters the pen, and only an exemplary illustration is provided here.

[0049] In block 220, the control terminal 110 may determine that the target vehicle 130 enters the pen in response to the relative position of the ground marking and the target vehicle 130 satisfying a preset condition. For example, the control terminal 110 may determine that the target vehicle 130 enters the pen in response to the parking orientation of the target vehicle 130 relative to the ground marking satisfying a preset angle (for example, 75 degrees - 90 degrees).

[0050] In block 230, the control terminal 110 may determine that the target vehicle 130 does not enter the pen in response to the relative position of the ground marking and the target vehicle 130 not satisfying the preset condition. For example, the control terminal 110 may determine that the target vehicle 130 does not enter the pen in response to the parking orientation of the target vehicle 130 relative to the ground marking not satisfying the preset angle (for example, 75 degrees - 90 degrees).

[0051] Additionally, the control terminal 110 may determine configuration information associated with the return area. As an example, the configuration information may indicate whether it is necessary to identify the damage degree of the ground marking in this return area.

[0052] As an example, the configuration information may be configured by relevant technical personnel (for example, the operation and maintenance personnel responsible for maintaining the ground marking) themselves.

[0053] Alternatively, the configuration information can also be periodically adjusted based on a predetermined duration (e.g., one day or one week, etc.). For example, the control terminal 110 can adjust the configuration information to the on-identification state (e.g., corresponding to turning on the parking line identification) after the duration of the configuration information of the return area being in the off-identification state (e.g., corresponding to turning off the parking line identification) reaches one week. The on-identification state can indicate that it is necessary to identify the damage degree of the ground markings in the return area. Further, the control terminal 110 can adjust the configuration information from the on-identification state to the off-identification state after determining the damage degree of the ground markings in the return area. The off-identification state can indicate that it is not necessary to identify the damage degree of the ground markings in the return area.

[0054] Alternatively, the target vehicle 130 entering or not entering the railing can respectively correspond to different configuration information. For example, when the target vehicle 130 enters the railing, the configuration information of the return area can be in the off-identification state. For example, when the target vehicle 130 does not enter the railing, the configuration information of the return area can be in the on-identification state. In this way, the embodiments of the present disclosure can confirm whether the failure to enter the railing is caused by the damage of the ground markings when the target vehicle 130 does not enter the railing.

[0055] In block 225, the control terminal 110 can end the railing entry identification process and normally proceed with the return process (e.g., complete the return) in response to the target vehicle 130 entering the railing and the configuration information being in the off-identification state (e.g., corresponding to turning off the parking line identification).

[0056] In block 235, the control terminal 110 can end the railing entry identification process and normally proceed with the return process (e.g., remind the user that the return fails) in response to the target vehicle 130 not entering the railing and the configuration information being in the off-identification state (e.g., corresponding to turning off the parking line identification).

[0057] In some embodiments, the control terminal 110 or the target vehicle 130 can determine the damage degree of the ground markings associated with the return area based on the target image. As an example, the damage of the ground markings can include, for example, wear, fading, peeling, cracking, deformation, missing, blurring, aging, pollution, structural damage, etc.

[0058] In some embodiments, after the railing entry identification process ends (e.g., the target vehicle 130 enters or does not enter the railing), the control terminal 110 or the target vehicle 130 can determine the damage degree of the ground markings associated with the return area based on the target image in response to the configuration information indicating that it is necessary to identify the damage degree of the ground markings in the return area.

[0059] As an example, the control terminal 110 or the shared vehicle 130 can provide the target image to a preset model to determine the damage degree of the ground markings associated with the return area.

[0060] As an example, the preset model can be configured on the target vehicle 130 or the user device 120 (the target vehicle 130 or the user device 120 can also be referred to as the edge side). As an example, whether the preset model is configured on the target vehicle 130 can be determined based on the hardware capabilities of the target vehicle 130 (for example, the computing power of the chip installed). Alternatively, the magnitude of the preset model can be determined based on the hardware capabilities of the target vehicle 130.

[0061] In block 240, when the preset model is configured on the target vehicle 130, the target vehicle 130 can store the target image in the local storage of the target vehicle 130.

[0062] In block 250, the target vehicle 130 can determine the damage degree (or state) of the ground marking (such as a parking line) based on the preset model.

[0063] As an example, the preset model can be configured at the control terminal 110 (or referred to as the server side, cloud, etc.) communicatively connected to the target vehicle 130.

[0064] In block 245, when the preset model is configured at the control terminal 110, the control terminal 110 can obtain the target image uploaded by the target vehicle 130.

[0065] In block 255, the control terminal 110 (also known as the cloud) can determine the damage degree (or state) of the ground marking (such as a parking line) based on the preset model.

[0066] In some embodiments, the preset model can be configured on the target vehicle 130 and at the control terminal 110 to respectively determine the damage degree of the ground marking and mutually verify the determination results.

[0067] In some embodiments, the target vehicle 130 or the control terminal 110 can provide the target image to the preset model configured at the control terminal 110 in response to the current network transmission rate being greater than a preset threshold, so that the preset model configured at the control terminal 110 can determine the damage degree of the ground marking. As an example, the target vehicle 130 or the control terminal 110 can provide the target image to the preset model configured on the target vehicle 110 in response to the current network transmission rate being less than the preset threshold, so that the preset model configured on the target vehicle 110 can determine the damage degree of the ground marking.

[0068] To better understand the above process of detecting the damage degree of the ground marking using the preset model, the following will further describe the process of Figures 3A to 3D detecting the ground marking. It should be understood that the example operations discussed in Figures 3A to 3D should not be construed as a limitation to the present disclosure.

[0069] As an example, the preset model may include a first classification model. As an example, the first classification model may be implemented as a machine learning model capable of outputting the damage degree of the ground marking based on an image. The present disclosure is not intended to limit the specific implementation and training process of this machine learning model.

[0070] As Figure 3A shown, Figure 3A a target image 300A is shown. The target image 300A includes a ground marking. The ground marking includes, for example, a parking line marking 305 associated with a preset color (such as a single-color parking line, a blue-white double-color line, a red-white double-color line, or a cyan-white double-color line, etc.). The target image 300A also includes a partial body image 306 associated with the target vehicle 130.

[0071] The first classification model may output a target classification label among a plurality of preset classification labels based on the target image 300A. The plurality of preset classification labels may correspond to a plurality of damage degrees of the ground marking. For example, the plurality of damage degrees of the ground marking may include no ground marking, intact ground marking, slightly damaged ground marking (for example, the ratio of the damaged area of the ground marking in the target image 300A to the total area of the region corresponding to the ground marking is less than a preset threshold (such as 0.5)), and severely damaged ground marking (for example, the ratio of the damaged area of the ground marking in the target image 300A to the total area of the region corresponding to the ground marking is greater than a preset threshold (such as 0.5)). Exemplarily, label 0 may correspond to no ground marking. Label 1 may correspond to an intact ground marking. Label 2 may correspond to a slightly damaged ground marking. Label 3 may correspond to a severely damaged ground marking.

[0072] As an example, the first classification model may output the target classification label as (1, 0.9) based on the target image 300A. As an example, 1 in (1, 0.9) may indicate that the ground marking associated with the return area is intact. 0.9 in (1, 0.9) may indicate that the confidence level of the intact ground marking is 0.9.

[0073] Alternatively or additionally, the target classification labels that the first classification model may output based on the target image may include (0, 0.02), (1, 0.9), (2, 0.03), and (3, 0.05). (0, 0.02) may indicate that the confidence level of no ground marking is 0.02. (1, 0.9) may indicate that the confidence level of an intact ground marking is 0.9. (2, 0.03) may indicate that the confidence level of a slightly damaged ground marking is 0.03. (3, 0.05) may indicate that the confidence level of a severely damaged ground marking is 0.05. Further, the control end 110 or the target vehicle 130 determines, based on the confidence level, that the damage degree of the ground marking in the return area is an intact ground marking.

[0074] In some embodiments, the preset model may include an object detection model and a second classification model. As an example, the object detection model may be implemented as a machine learning model capable of identifying ground markings in an image based on the image. The second classification model may be implemented as a machine learning model capable of outputting the degree of damage of the ground marking based on an image associated with the ground marking. The present disclosure is not intended to limit the specific implementation and training process of the object detection model and the second classification model.

[0075] As Figure 3B shown, Figure 3B FIG. shows a target image 300B. The target image 300B includes ground markings. The ground markings include, for example, parking line markings (such as markings 310-1, 310-2, and 310-3, etc.) associated with a preset shape combination (such as a green T line, etc.). The target image 300B also includes a partial body image 311 associated with the target vehicle 130.

[0076] As an example, the object detection model may determine, based on the target image 300B, the region associated with the ground marking in the target image 300B (such as the regions corresponding to markings 310-1, 310-2, and 310-3, etc.). The object detection model may, for example, output the category of the ground marking (such as 0 representing a parking line marking associated with a preset color combination, 1 representing a parking line marking associated with a preset shape, 2 representing a parking indication object), the position (such as position coordinates or a rectangular indication frame), and the confidence level. For example, the output result of the object detection model for the region where marking 310-1 is located may be (1, 0.79), indicating that marking 310-1 is a parking line marking associated with a preset shape (such as a green T line), and the confidence level of this output result is 0.79.

[0077] Additionally, the second classification model may determine the degree of damage of the ground marking based on the image content (such as a sub-image) associated with the region (such as the region associated with the ground marking determined by the object detection model) in the target image 300B.

[0078] As an example, the second classification model can output a target classification label among multiple preset classification labels based on the image content associated with a region in the target image 300B (e.g., the region associated with the ground sign determined by the object detection model). The multiple preset classification labels can correspond to multiple degrees of damage of the ground sign. For example, the multiple degrees of damage of the ground sign can include, for example, no ground sign, intact ground sign, slightly damaged ground sign (e.g., the ratio of the damaged area of the ground sign in the target image 300B to the total area of the region corresponding to the ground sign is less than a preset threshold (e.g., 0.5)), and severely damaged ground sign (e.g., the ratio of the damaged area of the ground sign in the target image 300B to the total area of the region corresponding to the ground sign is greater than a preset threshold (e.g., 0.5)). Exemplarily, label 0 can correspond to no ground sign. Label 1 can correspond to an intact ground sign. Label 2 can correspond to a slightly damaged ground sign. Label 3 can correspond to a severely damaged ground sign.

[0079] As an example, based on the image content of the region where the sign 310-1 is located in the target image 300B, the target classification labels output by the second classification model can include, for example, (0, 0.02), (1, 0.04), (2, 0.85), and (3, 0.09). (0, 0.02) can indicate that the confidence level of no ground sign is 0.02. (1, 0.04) can indicate that the confidence level of an intact ground sign is 0.04. (2, 0.85) can indicate that the confidence level of a slightly damaged ground sign is 0.85. (3, 0.09) can indicate that the confidence level of a severely damaged ground sign is 0.09. Further, the control end 110 or the target vehicle 130 determines, based on the confidence level, that the degree of damage of the ground sign in the return area is a slightly damaged ground sign.

[0080] Alternatively, the control end 110 or the target vehicle 130 can crop the target image 300B based on the region associated with the ground sign determined by the object detection model before the second classification model determines the degree of damage of the ground sign, so as to obtain a cropped target image (or called a sub-image). The cropped target image only retains the image part corresponding to the region associated with the ground sign. The second classification model can determine the degree of damage of the ground sign based on the cropped target image. In this way, the present disclosure determines the degree of damage of the ground sign based on the cropped target image, which can reduce the influence of the road surface background, the vehicle body, etc. on the recognition accuracy, thereby helping to improve the accuracy of determining the degree of damage of the ground sign.

[0081] Alternatively, the control end 110 or the target vehicle 130 can preprocess the cropped target image (e.g., normalization processing or image enhancement, etc.) so that the second classification model can determine the degree of damage of the ground sign based on the preprocessed target image, thereby improving the accuracy of the determination result.

[0082] In some embodiments, the preset model includes a segmentation model and a third classification model. As an example, the segmentation model can be implemented as a machine learning model capable of identifying ground markings in an image based on the image. The third classification model can be implemented as a machine learning model capable of outputting the degree of damage of the ground marking based on the semantic map associated with the ground marking. The present disclosure is not intended to limit the specific implementation and training process of the segmentation model and the third classification model.

[0083] As Figure 3C shown, Figure 3C a target image 300C is shown. The target image 300C includes ground markings. The ground markings include, for example, parking line markings 315-1 and 315-2 associated with a preset color (such as a single-color parking line, a blue-and-white double-color line, a red-and-white double-color line, or a cyan-and-white double-color line, etc.). The target image 300C also includes a partial body image 316 associated with the target vehicle 130.

[0084] As an example, the segmentation model can determine at least one region associated with the ground marking based on the target image 300C (such as a first region associated with the parking line marking 315-1 and a second region associated with the parking line marking 315-2). As an example, the segmentation model can be used to identify the ground marking in the target image 300C and determine the category (for example, 0 represents a parking line marking associated with a preset color combination, 1 represents a parking line marking associated with a preset shape, 2 represents a parking indication object), position (such as position coordinates or a rectangular indication box), and confidence level of the ground marking. For example, the output result of the target detection model for the region where the parking line marking 315-1 is located can be (0, 0.6), indicating that the parking line marking 315-1 is a parking line marking associated with a preset color combination, and the confidence level of this output result is 0.6. For example, the output result of the target detection model for the region where the parking line marking 315-2 is located can be (0, 0.75), indicating that the parking line marking 315-2 is a parking line marking associated with a preset color combination, and the confidence level of this output result is 0.75.

[0085] Additionally, the control terminal 110 or the target vehicle 130 can generate a semantic map corresponding to the target image based on the at least one region.

[0086] As an example, the semantic map is a highly abstract representation of the target image. Compared with using the target image itself, using the semantic map can directly exclude background image information unrelated to the ground marking at the input level, greatly reducing the dependence of the algorithm on the quantity and diversity of data, and significantly improving the anti-interference ability of the algorithm, and making the algorithm more interpretable.

[0087] As Figure 3D shown,Figure 3D The semantic map 300D corresponding to the target image 300C is shown. As an example, the semantic map 300D may include a first part 316-1 corresponding to the area where the stop line identification 315-1 is located and a second part 316-2 corresponding to the area where the stop line identification 315-2 is located.

[0088] As an example, the third classification model may determine the degree of damage associated with the ground identification based on the semantic map 300D. For example, the third classification model may output a target classification label among a plurality of preset classification labels based on the semantic map 300D. The plurality of preset classification labels may correspond to multiple degrees of damage of the ground identification. For example, the multiple degrees of damage of the ground identification may include no ground identification, the ground identification is intact, the ground identification is slightly damaged (for example, the ratio of the damaged area of the ground identification in the target image 300C to the total area of the region corresponding to the ground identification is less than a preset threshold (for example, 0.5)), and the ground identification is severely damaged (for example, the ratio of the damaged area of the ground identification in the target image 300C to the total area of the region corresponding to the ground identification is greater than a preset threshold (for example, 0.5)). Exemplarily, label 0 may correspond to no ground identification. Label 1 may correspond to the ground identification being intact. Label 2 may correspond to the ground identification being slightly damaged. Label 3 may correspond to the ground identification being severely damaged.

[0089] As an example, the target classification labels output by the third classification model based on the first part 316-1 in the semantic map 300D may include (0, 0.02), (1, 0.02), (2, 0.90), and (3, 0.05). (0, 0.02) may indicate that the confidence of no ground identification is 0.02. (1, 0.02) may indicate that the confidence of the ground identification being intact is 0.02. (2, 0.90) may indicate that the confidence of the ground identification being slightly damaged is 0.90. (3, 0.05) may indicate that the confidence of the ground identification being severely damaged is 0.05. Further, the control terminal 110 or the target vehicle 130 determines, based on the confidence, that the degree of damage of the ground identification in the return area is that the ground identification is slightly damaged.

[0090] Alternatively, the control terminal 110 or the target vehicle 130 may crop the semantic map 300D based on the region associated with the ground identification determined by the target detection model before the third classification model determines the degree of damage of the ground identification, so as to obtain a cropped semantic map. The cropped semantic map only retains the image part corresponding to the region associated with the ground identification. The third classification model may determine the degree of damage of the ground identification based on the cropped semantic map. In this way, the present disclosure determines the degree of damage of the ground identification based on the cropped semantic map, which can reduce the influence of the road surface background, the vehicle body, etc. on the recognition accuracy, thereby helping to improve the accuracy of determining the degree of damage of the ground identification.

[0091] Alternatively, the control terminal 110 or the target vehicle 130 may preprocess the cropped semantic map (e.g., normalization processing or image enhancement, etc.) so that the third classification model can determine the damage degree of the ground marking based on the preprocessed semantic map, thereby improving the accuracy of the determination result.

[0092] As an example, continue to refer to Figure 2 , after determining the damage degree of the ground marking associated with the return area, the control terminal 110 or the target vehicle 130 may query whether the return area (or referred to as the current location) needs to determine the damage degree of the ground marking (e.g., the status of the stop line).

[0093] Alternatively, at block 265, the control terminal 110 or the target vehicle 130 may end the process of determining the damage degree of the ground marking (e.g., the process of identifying the stop line marking) in response to the damage degree of the ground marking in the return area having been determined within a preset time period (e.g., today) or the current time period not requiring the determination of the damage degree of the ground marking.

[0094] Alternatively, at block 270, the control terminal 110 or the target vehicle 130 may execute a corresponding process based on the damage degree of the ground marking determined by a preset model in response to the damage degree of the ground marking in the return area not having been determined within a preset time period (e.g., today) and the current time period requiring the determination of the damage degree of the ground marking.

[0095] For example, at block 275, the control terminal 110 or the target vehicle 130 may end the process of determining the damage degree of the ground marking (e.g., the process of identifying the stop line marking) in response to the damage degree of the ground marking being less than or equal to a threshold value (e.g., the ground marking is not damaged or slightly damaged).

[0096] For example, at block 280, the control terminal 110 or the target vehicle 130 may send a notice associated with maintaining the ground marking in response to the damage degree of the ground marking being greater than the threshold value (e.g., the ground marking is severely damaged). For example, the control terminal 110 or the target vehicle 130 may query the location information reported by the return location (e.g., the return area) and notify the operation and maintenance (e.g., operation and maintenance personnel) to handle it at this location.

[0097] Based on the processes described above, embodiments of the present disclosure can determine the damage degree of a ground marking associated with a return area by detecting a target image of the return area corresponding to a vehicle return request, and further determine the damage degree of the ground marking associated with the return area from the target image. Further, embodiments of the present disclosure can send a notice associated with maintaining the ground marking in response to the damage degree of the ground marking being greater than a threshold. In this way, embodiments of the present disclosure can determine the damage degree of the ground marking during the vehicle return process without the need for an operation and maintenance personnel to go to the site to determine, thereby improving the efficiency of maintaining the ground marking, saving human resource costs, meeting the vehicle return needs of users, and improving the vehicle return experience of users.

[0098] Example process

[0099] Figure 4 FIG. 7 shows a flowchart of a process 400 for maintaining a ground marking according to some embodiments of the present disclosure.

[0100] It should be understood that the process 400 can be implemented on the service side (e.g., the control end 110), or can be implemented on the terminal side (e.g., at the user device 120 or the target vehicle 130, where the user device 120 or the target vehicle 130 includes functional modules for implementing the process 400).

[0101] Next, for the sake of convenience of description only, the process 400 will be described by taking the process 400 at the control end 110 as an example.

[0102] In block 410, the control end 110 obtains a target image associated with the return area in response to a vehicle return request for the target vehicle.

[0103] In block 420, the control end 110 determines the damage degree of the ground marking associated with the return area based on the target image.

[0104] In block 430, the control end 110 sends a notice associated with maintaining the ground marking in response to the damage degree of the ground marking being greater than a threshold.

[0105] In some embodiments, determining the damage degree of the ground marking associated with the return area based on the target image includes: providing the target image to a preset model to determine the damage degree of the ground marking associated with the return area.

[0106] In some embodiments, the preset model includes a first classification model, and the first classification model is used to output a target classification label among a plurality of preset classification labels based on the target image, and the plurality of preset classification labels correspond to a plurality of damage degrees of the ground marking.

[0107] In some embodiments, the preset model includes an object detection model and a second classification model. Determining the damage degree of the ground identifier associated with the vehicle return area includes: using the object detection model to determine the area associated with the ground identifier in the target image; and using the second classification model to process the sub-image associated with the area in the target image to determine the damage degree of the ground identifier.

[0108] In some embodiments, the preset model includes a segmentation model and a third classification model. Determining the damage degree of the ground identifier associated with the vehicle return area includes: using the segmentation model to process the target image to determine at least one area associated with the ground identifier; generating a semantic map corresponding to the target image based on the at least one area; and the third classification model determining the damage degree associated with the ground identifier based on the semantic map.

[0109] In some embodiments, the preset model is configured on the target vehicle or at a server in communication connection with the target vehicle.

[0110] In some embodiments, determining the damage degree of the ground identifier associated with the vehicle return area based on the target image includes: determining configuration information associated with the vehicle return area, where the configuration information indicates whether it is necessary to identify the damage degree of the ground identifier in the vehicle return area; and in response to the configuration information indicating that it is necessary to identify the damage degree of the ground identifier in the vehicle return area, determining the damage degree of the ground identifier associated with the vehicle return area based on the target image.

[0111] In some embodiments, the ground identifier includes at least one of the following: a stop line identifier associated with a preset color; a stop line identifier associated with a preset shape.

[0112] In some embodiments, obtaining the target image associated with the vehicle return area includes: using an image acquisition device deployed on the target vehicle to obtain the target image associated with the vehicle return area.

[0113] Example devices and equipment

[0114] Figure 5 FIG. shows a schematic structural block diagram of a device 500 for maintaining a ground identifier according to some embodiments of the present disclosure. The device 500 can be implemented as or included in the control terminal 110 or the user device 120. Each module / component in the device 500 can be implemented by hardware, software, firmware, or any combination thereof.

[0115] As Figure 5As shown, device 500 includes an acquisition module 510 configured to acquire a target image associated with a return area in response to a return request for a target vehicle; a determination module 520 configured to determine the damage degree of a ground identifier associated with the return area based on the target image; and a sending module 530 configured to send a notice associated with maintaining the ground identifier in response to the damage degree of the ground identifier being greater than a threshold value.

[0116] In some embodiments, the determination module 520 is further configured to: provide the target image to a preset model to determine the damage degree of the ground identifier associated with the return area.

[0117] In some embodiments, the preset model includes a first classification model, and the first classification model is used to output a target classification label among a plurality of preset classification labels based on the target image, and the plurality of preset classification labels correspond to multiple damage degrees of the ground identifier.

[0118] In some embodiments, the preset model includes a target detection model and a second classification model, wherein the determination module 520 is further configured to: use the target detection model to determine the area associated with the ground identifier in the target image; and use the second classification model to process the sub-image associated with the area in the target image to determine the damage degree of the ground identifier.

[0119] In some embodiments, the preset model includes a segmentation model and a third classification model, wherein the determination module 520 is further configured to: process the target image using the segmentation model to determine at least one area associated with the ground identifier; generate a semantic map corresponding to the target image based on the at least one area; and the third classification model determines the damage degree associated with the ground identifier based on the semantic map.

[0120] In some embodiments, the preset model is configured on the target vehicle or at a server in communication connection with the target vehicle.

[0121] In some embodiments, the determination module 520 is further configured to: determine configuration information associated with the return area, where the configuration information indicates whether it is necessary to identify the damage degree of the ground identifier in the return area; and in response to the configuration information indicating that it is necessary to identify the damage degree of the ground identifier in the return area, determine the damage degree of the ground identifier associated with the return area based on the target image.

[0122] In some embodiments, the ground identifier includes at least one of the following: a parking line identifier associated with a preset color; a parking line identifier associated with a preset shape.

[0123] In some embodiments, the acquisition module 510 is further configured to: use an image acquisition device deployed on the target vehicle to acquire a target image associated with the return area.

[0124] The modules included in apparatus 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or in place of the machine-executable instructions, some or all of the modules in apparatus 500 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0125] Figure 6 A block diagram of an electronic device 600 in which one or more embodiments of the present disclosure can be implemented is shown. It should be understood that Figure 6 the illustrated electronic device 600 is merely exemplary and should not impose any limitation on the functionality and scope of the embodiments described herein. The electronic device 600 can include or be implemented as Figure 5 the apparatus 500.

[0126] As Figure 6 shown, the electronic device 600 is in the form of a general-purpose computing device. The components of the electronic device 600 can include, but are not limited to, one or more processors or processing units 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 can be an actual or virtual processor and is capable of performing various processes according to the programs stored in the memory 620. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing ability of the electronic device 600.

[0127] The electronic device 600 generally includes multiple computer storage media. Such media can be any available media accessible to the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 can be volatile memory (such as registers, caches, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 630 can be removable or non-removable media and can include machine-readable media, such as flash drives, magnetic disks, or any other media that can be used to store information and / or data (such as training data for training) and can be accessed within the electronic device 600.

[0128] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 6 , a disk drive for reading from and writing to a removable, non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading from and writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 620 may include a computer program product 625 having one or more program modules that are configured to perform the various methods or actions of the various embodiments of the present disclosure.

[0129] The communication unit 640 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 600 may be implemented in a single computing cluster or multiple computer machines that are capable of communicating via a communication connection. Thus, the electronic device 600 may operate in a networked environment using a logical connection to one or more other servers, network personal computers (PCs), or another network node.

[0130] The input device 650 may be one or more input devices, such as a mouse, keyboard, trackball, etc. The output device 660 may be one or more output devices, such as a display, speaker, printer, etc. The electronic device 600 may also communicate with one or more external devices (not shown) as needed via the communication unit 640, such as a storage device, a display device, etc., communicate with one or more devices that enable a user to interact with the electronic device 600, or communicate with any device that enables the electronic device 600 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0131] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, where the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, the computer program product being tangibly stored on a non-transitory computer-readable medium and including computer-executable instructions, and the computer-executable instructions being executed by a processor to implement the method described above.

[0132] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0133] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, when executed by the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises a manufacture including instructions that implement various aspects of the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0134] The computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process, such that the instructions executed on the computer, other programmable data processing apparatus, or other devices implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0135] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by special-purpose hardware-based systems that perform the specified functions or acts, or by combinations of special-purpose hardware and computer instructions.

[0136] The implementations of the present disclosure have been described above. The description is exemplary, not exhaustive, and is not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The choice of terms used herein is intended to best explain the principles of the implementations, the practical application, or the improvement of technologies in the market, or to enable other ordinary skilled persons in the art to understand the implementations disclosed herein.

Claims

1. A method for maintaining ground markings, comprising: In response to a vehicle return request for a target vehicle, acquiring a target image associated with a vehicle return area; Determining, based on the target image, a degree of damage of a ground sign associated with the vehicle return area; and In response to the damage degree of the ground sign being greater than a threshold, sending a notification associated with maintaining the ground sign.

2. The method according to claim 1, wherein determining the degree of damage of the ground sign associated with the vehicle return area based on the target image comprises: The target image is provided to a preset model to determine the degree of damage to the ground marking associated with the vehicle return area.

3. The method according to claim 2, wherein the preset model includes a first classification model, and the first classification model is used to output a target classification label from a plurality of preset classification labels based on the target image, and the plurality of preset classification labels correspond to a plurality of damage degrees of the ground mark.

4. The method according to claim 2, wherein the preset model comprises a target detection model and a second classification model, wherein determining the degree of damage of the ground sign associated with the vehicle return area comprises: Determining, using the target detection model, an area in the target image that is associated with the ground marker; as well as The sub-image associated with the area in the target image is processed using the second classification model to determine the degree of damage of the ground mark.

5. The method according to claim 2, wherein the preset model comprises a segmentation model and a third classification model, wherein determining the damage degree of the ground mark associated with the vehicle return area comprises: Processing the target image using the segmentation model to determine at least one region associated with the ground marker; Based on the at least one region, generating a semantic map corresponding to the target image; as well as The third classification model determines the degree of damage associated with the ground mark based on the semantic graph. 6 . The method according to claim 2 , wherein the preset model is configured on the target vehicle or at a service end that is communicatively connected to the target vehicle.

7. The method according to claim 1, wherein determining the degree of damage of the ground sign associated with the vehicle return area based on the target image comprises: Determining configuration information associated with the vehicle return area, the configuration information indicating whether it is necessary to identify the degree of damage of the ground mark of the vehicle return area; as well as In response to the configuration information indicating that a damage degree of the ground sign of the vehicle return area needs to be identified, the damage degree of the ground sign associated with the vehicle return area is determined based on the target image.

8. The method according to claim 1, wherein the ground mark comprises at least one of the following: Stop line markings associated with preset colors; Stop line markings associated with preset shapes.

9. The method of claim 1, wherein acquiring a target image associated with a vehicle return area comprises: The target image associated with the vehicle return area is acquired by using an image acquisition device deployed on the target vehicle.

10. A device for maintaining ground markings, comprising: an acquisition module configured to acquire a target image associated with a vehicle return area in response to a vehicle return request for a target vehicle; a determination module configured to determine the degree of damage of a ground sign associated with the vehicle return area based on the target image; as well as The sending module is configured to send a notification associated with maintaining the ground sign in response to the damage degree of the ground sign being greater than a threshold.

11. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 9 when executed by the at least one processing unit.

12. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 9.

Images