A rod-shaped object recognition method, device, equipment and storage medium

Through an image segmentation network based on semantic segmentation, the precise position of the rod object is identified, which solves the problem of inaccurate position recognition of rod object in the prior art, and improves the positioning accuracy and safety of autonomous driving.

CN115294552BActive Publication Date: 2024-11-05TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210945907.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-08
Publication Date
2024-11-05
Estimated Expiration
2042-08-08

AI Technical Summary

Technical Problem

The existing rod object recognition methods cannot accurately obtain the precise position information of rod object, resulting in errors in position correction in high-precision maps, affecting the positioning accuracy of autonomous driving.

Method used

An image segmentation network based on semantic segmentation is used to obtain the image to be identified and perform segmentation network training to obtain the segmentation result of the rod-shaped object, and a set of center points is obtained by scanning, and the center line is calculated to determine the position information of the rod-shaped object.

Benefits of technology

It realizes accurate position recognition of rod-shaped objects, improves the accuracy of high-precision maps, and thus improves the positioning accuracy and safety of autonomous driving.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115294552B_ABST
    Figure CN115294552B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide a rod-shaped object recognition method, device, equipment and storage medium for obtaining accurate position information of rod-shaped objects. The method includes: obtaining an image to be recognized, the image to be recognized includes at least one rod-shaped object to be recognized; inputting the image to be recognized into a segmentation network to obtain a first segmentation result, the first segmentation result is a segmentation result of the first rod-shaped object to be recognized in the image to be recognized, the segmentation network is an image segmentation network based on semantic segmentation obtained by training based on labeled training images; scanning the first segmentation result to obtain a first center point set corresponding to the first rod-shaped object to be recognized, the first center point set includes the center point of the first rod-shaped object to be recognized obtained by scanning; obtaining a first center line corresponding to the first rod-shaped object to be recognized based on the center point in the first center point set; determining the position information of the first rod-shaped object to be recognized based on the first center line. The present application can be applied to the fields of transportation, mapping and Internet of Vehicles.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to a rod-shaped object recognition method, device, equipment and storage medium. Background Art

[0002] High-precision maps are electronic maps with higher accuracy and more data dimensions. The construction of high-precision maps is considered an important technical link in autonomous driving technology. High-precision maps contain a large amount of information that can be used as a reference for autonomous vehicles during driving, such as road information such as the location, type, width, slope and curvature of lane lines, and information about fixed objects around lanes such as traffic signs, traffic lights, guardrails, road edges, poles, etc. Figure 1 An important application is high-precision positioning. Autonomous driving requires vehicles to have centimeter-level positioning accuracy, while the positioning of the Global Positioning System (GPS) has errors of meters or even tens of meters. At this time, the position can be corrected in combination with elements of high-precision maps. Since poles are common static targets in road scenes, they are usually used as an important reference element in high-precision maps.

[0003] In the perception of rod-shaped objects in images, it is currently usually based on target detection. That is, an external rectangular frame is output for the rod-shaped object. The detection rectangular frame can only output the area containing the rod-shaped object, so that the position information range of the rod-shaped object in the image is large, and the precise position information of the rod-shaped object cannot be identified. Summary of the invention

[0004] The embodiments of the present application provide a rod-shaped object recognition method, device, equipment and storage medium for recognizing precise position information of a rod-shaped object.

[0005] In view of this, on the one hand, the present application provides a rod-shaped object recognition method, comprising: acquiring an image to be recognized, the image to be recognized including at least one rod-shaped object to be recognized; inputting the image to be recognized into a segmentation network to obtain a first segmentation result, the first segmentation result being a segmentation result of a first rod-shaped object to be recognized in the image to be recognized, the segmentation network being an image segmentation network based on semantic segmentation obtained by training according to training image data and an initial segmentation network; scanning the first segmentation result to obtain a first center point set corresponding to the first rod-shaped object to be recognized, the first center point set including the center point of the first rod-shaped object to be recognized obtained after scanning; obtaining a first midline corresponding to the first rod-shaped object to be recognized according to the center point in the first center point set; and determining the position information of the first rod-shaped object to be recognized according to the first midline.

[0006] Another aspect of the present application provides a rod-shaped object recognition device, comprising:

[0007] an acquisition module, for acquiring an image to be recognized, the image to be recognized comprising at least one rod-shaped object to be recognized; inputting the image to be recognized into a segmentation network to obtain a first segmentation result, the first segmentation result being a segmentation result of the first rod-shaped object to be recognized in the image to be recognized, the segmentation network being an image segmentation network based on semantic segmentation obtained by training the training image data and the initial segmentation network;

[0008] A processing module, configured to scan the first segmentation result to obtain a first center point set corresponding to the first rod-shaped object to be identified, wherein the first center point set includes center points of the first rod-shaped object to be identified obtained by scanning; and obtain a first midline corresponding to the first rod-shaped object to be identified according to the center points in the first center point set;

[0009] The determination module is used to determine the position information of the first rod-shaped object to be identified according to the first center line.

[0010] In one possible design, in another implementation of another aspect of the embodiment of the present application, the processing module is specifically used to perform pixel scanning on the first segmentation result row by row to determine a first horizontal coordinate and a second horizontal coordinate of the first segmentation result on each row, wherein the first horizontal coordinate is the horizontal coordinate of the first appearance position of the first segmentation result in each row, and the second horizontal coordinate is the horizontal coordinate of the last appearance position of the first segmentation result in each row; taking the middle value of the first horizontal coordinate and the second horizontal coordinate to obtain the horizontal coordinate of the first center point of the first segmentation result in each row, and then taking the vertical coordinate of each row as the vertical coordinate of the first center point, and generating the first center point set based on the first center point.

[0011] In one possible design, in another implementation of another aspect of the embodiment of the present application, the processing module is specifically used to segment the first segmentation result according to a preset number of rows to obtain a scanning line; perform pixel scanning on the scanning line to determine a third horizontal coordinate and a fourth horizontal coordinate of the first segmentation result on the scanning line, wherein the third horizontal coordinate is the horizontal coordinate of the first appearance position of the first segmentation result in the scanning line, and the fourth horizontal coordinate is the horizontal coordinate of the last appearance position of the first segmentation result in the scanning line; take the middle value of the third horizontal coordinate and the fourth horizontal coordinate to obtain the horizontal coordinate of the second center point of the first segmentation result in the scanning line, and then take the vertical coordinate of the scanning line as the vertical coordinate of the second center point, and generate the first center point set based on the second center point.

[0012] In a possible design, in another implementation of another aspect of the embodiment of the present application, the processing module is specifically used to determine the horizontal coordinate variation range corresponding to the first center point set according to the horizontal coordinate of each center point in the first center point set;

[0013] When the horizontal coordinate variation range is less than or equal to the first preset threshold, the first center line is defined as a vertical center line; wherein the horizontal coordinate of the first center line is the average of the horizontal coordinates of each center point in the first center point set, and the vertical coordinate of the first center line is the minimum vertical coordinate and the maximum vertical coordinate of each center point in the first center point set;

[0014] When the horizontal coordinate variation range is greater than the first preset threshold value and less than the second preset threshold value, performing fitting processing on each center point of the first center point set to obtain the first center line;

[0015] When the horizontal coordinate variation range is greater than or equal to the second preset threshold, the abnormal center point is discarded from the first center point set to obtain a second center point set, and the center points of the second center point set are fitted to obtain the first center line, and the abnormal center point is a center point whose horizontal coordinate difference with the average of the horizontal coordinates of each center point in the first center point set exceeds the third preset threshold.

[0016] In one possible design, in another implementation of another aspect of the embodiment of the present application, the acquisition module is further used to acquire the training image data and the initial segmentation network, the training image data is annotated with rods, the initial segmentation network adopts an encoder-decoder structure, and the initial segmentation network includes a context information enhancement module for acquiring vertical information of the rods;

[0017] The rod-shaped object recognition device also includes a training module, which is used to train the initial segmentation network according to the training image data to obtain the segmentation network.

[0018] In one possible design, in another implementation of another aspect of the embodiment of the present application, the context information enhancement module is a plurality of vertical asymmetric convolution layers of different scales, and the convolution kernel of the vertical asymmetric convolution layer is designed to be N*1, where N is a positive integer greater than 1; the convolution module of the encoder is a lightweight depthwise-pointwise convolution module.

[0019] In a possible design, in another implementation of another aspect of the embodiment of the present application, the acquisition module is specifically used to output the image to be identified to the segmentation network to output M second segmentation results, the M second segmentation results are segmentation results of all rod-shaped objects to be identified in the image to be identified, and M is an integer greater than or equal to 1;

[0020] An opening operation is performed on the M second segmentation results so that adhesions existing in the M second segmentation results are disconnected to obtain X independent block segmentation results, and the first segmentation result of the first rod-shaped object to be identified is included in the X independent block segmentation results, where X is an integer greater than or equal to M.

[0021] Another aspect of the present application provides a computer device, including: a memory, a processor, and a bus system;

[0022] Wherein, the memory is used to store programs;

[0023] The processor is used to execute the program in the memory, and the processor is used to execute the above-mentioned methods according to the instructions in the program code;

[0024] The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

[0025] Another aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer is enabled to execute the above-mentioned methods.

[0026] Another aspect of the present application provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided by the above aspects.

[0027] It can be seen from the above technical scheme that the embodiments of the present application have the following advantages: after obtaining a prediction result based on semantic segmentation of the rod-shaped object in the image to be identified, a set of center points of the rod-shaped object is calculated according to the segmentation result, and then the midline of the rod-shaped object is obtained, and finally the position information of the rod-shaped object in the image to be identified is determined according to the midline, thereby obtaining the precise position information of the rod-shaped object. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of the architecture of a communication system in an embodiment of the present application;

[0029] Figure 2 This is an example diagram of the application of the position information of the rod-shaped object in the process of autonomous driving in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of an embodiment of a rod-shaped object recognition method in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of an image to be recognized in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of a segmentation result output by a segmentation network of an image to be identified in an embodiment of the present application;

[0033] Figure 6 In the embodiments of this application, based on Figure 5 A schematic diagram of the segmentation result corresponding to the image to be recognized after adjustment;

[0034] Figure 7 A schematic diagram of a structure of a segmented network in an embodiment of the present application;

[0035] Figure 8 A schematic diagram of a set of center points of a rod-shaped object to be identified in an embodiment of the present application;

[0036] Fig. 9 Another schematic diagram of a set of center points of a rod-shaped object to be identified in an embodiment of the present application;

[0037] Fig.10 In the embodiments of this application, based on Figure 6 Another schematic diagram of a set of center points of the segmentation result shown;

[0038] Fig.11 In the embodiments of this application, based on Fig.10 A schematic diagram of the midline of the rod to be identified is shown;

[0039] Fig.12 This is a schematic diagram of an embodiment of a rod-shaped object recognition device in an embodiment of the present application;

[0040] Fig.12a This is a schematic diagram of an embodiment of a rod-shaped object recognition device in an embodiment of the present application;

[0041] Fig.13 This is a schematic diagram of another embodiment of the rod-shaped object recognition device in the embodiment of the present application;

[0042] Fig.14 FIG. 1 is a schematic diagram of another embodiment of the rod-shaped object recognition device in the embodiment of the present application. DETAILED DESCRIPTION

[0043] The embodiments of the present application provide a rod-shaped object recognition method, device, equipment and storage medium for obtaining accurate position information of a rod-shaped object.

[0044] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0045] For ease of understanding, some terms in this application are explained below:

[0046] Pole-shaped object: In this application, it refers to the vertical poles that appear in traffic scenes, including traffic sign poles, traffic light poles, electric poles, isolation poles, etc.

[0047] Semantic segmentation: A typical computer vision problem that involves taking some raw data (e.g., flat images) as input and converting them into masks with highlighted regions of interest. Many people use the term full-pixel semantic segmentation, where each pixel in the image is assigned a class ID based on the object of interest to which it belongs. Early computer vision problems only found elements such as edges (lines and curves) or gradients, but they never fully provided pixel-level image understanding in the way humans perceive. Semantic segmentation solves this problem by grouping together parts of an image that belong to the same object, thereby expanding its application areas. Its classification includes: standard semantic segmentation, also known as full-pixel semantic segmentation, is the process of classifying each pixel as belonging to an object class; instance aware semantic segmentation is a subtype of standard semantic segmentation or full-pixel semantic segmentation, which classifies each pixel as belonging to an object class and the identity of the entity in that class.

[0048] Mask: Also called image mask. Image masking refers to recalculating the value of each pixel in the image through the mask kernel operator. The mask kernel operator describes the influence of the domain pixel point on the new pixel value, and performs weighted averaging on the pixel points according to the weight factor in the mask operator. Image masking is often used in areas such as image smoothing, edge detection, and feature analysis.

[0049] The method provided in this application is applied to Figure 1 The communication system shown is shown in Figure 1 , Figure 1 is a schematic diagram of the architecture of the communication system in the embodiment of the present application, such as Figure 1 As shown, the communication system includes a server, a terminal device and a satellite, and the client is deployed on the terminal device, wherein the client can be run on the terminal device in the form of a browser, or can be run on the terminal device in the form of an independent application (application, APP), etc. The specific presentation form of the client is not limited here. The server involved in this application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal device can be a smart phone, a tablet computer, a laptop, a PDA, a personal computer, a smart TV, a smart watch, a vehicle-mounted device, a wearable device, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in this application. The number of servers and terminal devices is also not limited. The solution provided in this application can be completed independently by a terminal device, can be completed independently by a server, or can be completed by a terminal device and a server in cooperation. This application does not make any specific limitations on this.

[0050] The terminal device obtains Global Positioning System (GPS) data. Usually, the terminal device can obtain GPS data from one or more GPS satellites while being outdoors and having line-of-sight access to the satellite. The terminal device uses GPS data to periodically determine GPS coordinates (e.g., longitude and latitude coordinates) to indicate the current location of the terminal device. Autonomous driving requires vehicles to have centimeter-level positioning accuracy, while GPS positioning has meter-level or even ten-meter-level errors. At this time, the position can be corrected in combination with elements of high-precision maps. High-precision maps are electronic maps with higher accuracy and more data dimensions. The construction of high-precision maps is considered an important technical link in autonomous driving technology. High-precision maps contain a large amount of information that can be used as a reference for autonomous driving vehicles during driving, such as road information such as the location, type, width, slope and curvature of lane lines, and information on fixed objects around lanes such as traffic signs, traffic lights, guardrails, road edges, and poles. An exemplary scenario such as Figure 2As shown, during driving, the vehicle perceives lane lines, traffic signs, poles and other objects through cameras and vehicle-side visual perception algorithms, and then compares the positional relationship between the vehicle and these objects with the data stored in the high-precision map to infer the precise vehicle position. Since poles are common static targets in road scenes, they are usually used as an important reference element in high-precision maps. In the perception of poles in images, it is currently usually based on target detection. That is, an external rectangular frame is output for the pole-shaped target. The detection rectangular frame can only output the area containing the pole-shaped object, which makes the position information range of the pole-shaped object in the image larger and the precise position information of the pole-shaped object cannot be identified.

[0051] In order to solve this problem, the present application provides the following technical solutions: obtaining an image to be identified, the image to be identified includes at least one rod-shaped object to be identified; inputting the image to be identified into a segmentation network to obtain a first segmentation result, the first segmentation result is the segmentation result of the first rod-shaped object to be identified in the image to be identified, the segmentation network is an image segmentation network based on semantic segmentation obtained by training according to training image data and an initial segmentation network; scanning the first segmentation result to obtain a first center point set corresponding to the first rod-shaped object to be identified, the first center point set includes the center points of the first rod-shaped object to be identified obtained by scanning; obtaining a first midline corresponding to the first rod-shaped object to be identified according to the center points in the first center point set; determining the position information of the first rod-shaped object to be identified according to the first midline. In this way, after obtaining a prediction result based on semantic segmentation for the rod-shaped object in the image to be identified, the center point set of the rod-shaped object is calculated according to the segmentation result, and then the midline of the rod-shaped object is obtained, and finally the position information of the rod-shaped object in the image to be identified is determined according to the midline, thereby obtaining the precise position information of the rod-shaped object. The technical solution provided in this application can be applied to the construction of high-precision maps, thereby realizing the application in autonomous driving. The precise position of the rod-shaped object is provided during the road perception process, thereby making the high-precision map more accurate and thus making autonomous driving safer.

[0052] In the present application, the position information of the rod-shaped object can be stored in a database. In short, a database can be regarded as an electronic file cabinet - a place to store electronic files, and users can add, query, update, delete, and other operations on the data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program. A database management system (DBMS) is a computer software system designed for managing databases, generally with basic functions such as storage, interception, security, and backup. Database management systems can be classified according to the database models they support, such as relational, Extensible Markup Language (XML); or according to the types of computers they support, such as server clusters and mobile phones; or according to the query language used, such as Structured Query Language (SQL), XQuery; or according to the performance impulse focus, such as maximum scale, maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, supporting multiple query languages ​​at the same time.

[0053] It is understandable that in the specific implementation of the present application, related data such as vehicle data and road information perceived by sensors are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0054] In combination with the above introduction, the following takes the terminal device as an example to introduce the rod-shaped object recognition method in this application. Figure 3 In the present application, one embodiment of the rod-shaped object recognition method includes:

[0055] 301. Acquire an image to be identified, where the image to be identified includes at least one rod-shaped object to be identified.

[0056] The terminal device obtains the image to be identified by the camera during the driving process of the vehicle. In this embodiment, the image to be identified can be a real-time image of the traffic road, so the image to be identified can include a variety of rod-shaped objects. In an exemplary solution, the image to be identified can be as follows Figure 4 As shown, it includes traffic sign poles, traffic light poles, and electric poles.

[0057] 302. Input the image to be identified into a segmentation network to obtain a first segmentation result, where the first segmentation result is a segmentation result of a first rod-shaped object to be identified in the image to be identified. The segmentation network is an image segmentation network based on semantic segmentation obtained by training the training image data and the initial segmentation network.

[0058] After acquiring the image to be identified, the terminal device inputs the image to be identified into a segmentation network deployed on the terminal device to obtain a second segmentation result of multiple rod-shaped objects in the image to be identified. At this time, since there may be overlapping parts among the rod-shaped objects in the image to be identified, the terminal device also needs to adjust the second segmentation result corresponding to the image to be identified so that the segmentation result corresponding to each rod-shaped object becomes an independent first segmentation result. The specific operation method can be as follows: the image to be identified is output to the segmentation network to output M segmentation results, where M is an integer greater than or equal to 1; the M second segmentation results are opened so that the adhesion existing in the M segmentation results is disconnected to obtain X independent segmentation results, and the M segmentation results and the X independent segmentation results are both used to indicate the segmentation results of the rod-shaped objects to be identified in the image to be identified, and the first segmentation result of the first rod-shaped object to be identified is included in the X independent segmentation results, where X is an integer greater than or equal to M.

[0059] In an exemplary embodiment, Figure 4 Explain that Figure 4 The image to be identified is input into the segmentation network and then output Figure 5 The predicted result graph shown contains the rod segmentation result. Figure 5 As can be seen from the figure, the segmentation results of the traffic sign pole and the traffic light pole partially overlap, so Figure 5 The segmentation results shown are 4 independent blocks. In order to separate the segmentation results of the traffic sign pole and the traffic light pole into independent blocks, Figure 5 The segmentation result shown is adjusted by opening operation to obtain Figure 6 The segmentation results of the five independent blocks are shown. At the same time, the segmentation results of the four independent blocks can also be numbered. For example, the segmentation result of the traffic sign pole is recorded as traffic sign pole 1, the segmentation result of the traffic light pole is recorded as traffic light pole 2, and the segmentation result of the electric pole is recorded as electric pole 3, electric pole 4 and electric pole 5.

[0060] In this embodiment, the segmentation network is set as an image segmentation network based on semantic segmentation, and the segmentation result can be output in the form of a pixel-level mask. It is understandable that the segmentation result can also be output in other pixel-level annotation forms, which are not specifically limited here.

[0061] Optionally, in order to ensure that more comprehensive feature information of the rod-shaped objects is extracted, the initial segmentation network adopts an encoder-decoder structure, and the initial segmentation network includes a contextual information enhancement module for obtaining the vertical information of the rod-shaped objects, wherein the contextual information enhancement module is a plurality of vertical asymmetric convolution layers of different scales, and the convolution kernel of the vertical asymmetric convolution layer is designed to be N*1. At the same time, in order to enable the segmentation network to be applied to terminal devices with poor computing power (such as vehicle-mounted terminals or smart phones), the convolution module of the encoder of the initial segmentation network can be designed as a lightweight depthwise-pointwise convolution module. In an exemplary scheme, the structure of the initial segmentation network can be as follows: Figure 7 As shown, the vertical information pyramid (i.e. the contextual information enhancement module of the vertical information mentioned above) is added between the encoder and the decoder in the initial segmentation network. Figure 7 As shown in FIG. 1 , the convolution module of the encoder is a lightweight depthwise-pointwise convolution module, the decoder includes an upsampling module, and the vertical information pyramid is a plurality of vertical asymmetric convolution layers of different scales. The convolution layers of the depthwise-pointwise convolution module can be as follows: Figure 7 As shown, it includes a convolution layer with a specification of "Conv, 1×1, Relu", a convolution layer with a specification of "Dwise3×3, Relu, stride=n", a convolution layer with a specification of "Conv, 1×1", a normalization layer (i.e., BatchNorm layer) and a fusion layer. The data processing flow in this depthwise-pointwise convolution module can be as follows: the image to be identified is sequentially passed through a convolution layer with a specification of "Conv, 1×1, Relu", a convolution layer with a specification of "Dwise3×3, Relu, stride=n", a convolution layer with a specification of "Conv, 1×1", a convolution layer with a specification of "Dwise3×3, Relu, stride=n" and the normalization layer to obtain an output result, and then the output result is fused with the image to be identified and processed through the ReLU function to obtain the output result output to the vertical information pyramid. The vertical information pyramid can be as follows Figure 7As shown, it includes three parallel convolution modules, one convolution module includes a convolution layer and a normalization layer with a specification of "Conv, 3×1"; another convolution module includes a convolution layer and a normalization layer with a specification of "Conv, 5×1"; another convolution module includes a convolution layer and a normalization layer with a specification of "Conv, 7×1". In this vertical information pyramid, the data processing flow is as follows: the output results of the depthwise-pointwise convolution module are respectively input into the above three convolution modules to obtain three different output results, and then the three output results are fused with the output results of the depthwise-pointwise convolution module and processed through the ReLU function to obtain the output results output to the decoder. The upsampling module in the decoder can be as follows Figure 7 The upsampling module includes a deconvolution layer and a normalization layer with a specification of "Transpose3×3, stride=2". The data processing flow of the upsampling module can be as follows: the output result of the vertical information pyramid is passed through the deconvolution layer and the normalization layer to obtain an upsampled output result, and the output result is processed through the ReLU function to obtain the segmentation result of the image to be identified.

[0062] 303. Scan the first segmentation result to obtain a first center point set corresponding to the first rod-shaped object to be identified, where the first center point set includes the center point of the first rod-shaped object to be identified obtained by scanning.

[0063] After the terminal device obtains the segmentation results of each rod-shaped object to be identified in the image to be identified, the segmentation results of each rod-shaped object to be identified are sequentially subjected to pixel scanning to obtain a center point, and the center line of each rod-shaped object to be identified is determined according to the center point, that is, the independent block segmentation results are respectively subjected to pixel scanning, and only one segmentation result is subjected to pixel scanning each time. In this embodiment, the first segmentation result corresponding to the first rod-shaped object to be identified is used as an example for explanation:

[0064] In a possible implementation, the terminal device performs pixel scanning on the first segmentation result row by row to determine the first horizontal coordinate and the second horizontal coordinate of the first segmentation result on each row, wherein the first horizontal coordinate is the horizontal coordinate of the first appearance position of the first segmentation result on each row, and the second horizontal coordinate is the horizontal coordinate of the last appearance position of the first segmentation result on each row, and then the middle value of the first horizontal coordinate and the second horizontal coordinate is taken to obtain the horizontal coordinate of the first center point of the first segmentation result on each row, and then the vertical coordinate of each row is taken as the vertical coordinate of the first center point, and the first center point set is generated according to the first center point. Figure 8As shown, on the first row, the first horizontal coordinate corresponding to the first segmentation result is 35, the second horizontal coordinate is 37, and the horizontal coordinate of the center point is 36. On the second row, the first horizontal coordinate corresponding to the first segmentation result is 34, the second horizontal coordinate is 36, and the horizontal coordinate of the center point is 35, and so on to obtain the center point of the first segmentation result on each row.

[0065] In another possible implementation, the terminal device segments the first segmentation result according to a preset number of lines to obtain a scan line; then performs pixel scanning on the scan line to determine the third horizontal coordinate and the fourth horizontal coordinate of the first segmentation result on the scan line, wherein the third horizontal coordinate is the horizontal coordinate of the first appearance position of the first segmentation result on the scan line, and the fourth horizontal coordinate is the horizontal coordinate of the last appearance position of the first segmentation result on the scan line; the middle value of the third horizontal coordinate and the fourth horizontal coordinate is taken to obtain the horizontal coordinate of the second center point of the first segmentation result on the scan line, and then the vertical coordinate of the scan line is taken as the vertical coordinate of the second center point, and the first center point set is generated based on the second center point. Fig. 9 As shown, the terminal device determines a scanning line every 20 lines on the image to be identified. On the first scanning line, the first horizontal coordinate corresponding to the first segmentation result is 35, the second horizontal coordinate is 37, and the horizontal coordinate of the center point is 36. On the second scanning line, the first horizontal coordinate corresponding to the first segmentation result is 34, the second horizontal coordinate is 36, and the horizontal coordinate of the center point is 35. The center point of the first segmentation result on each scanning line is obtained by analogy.

[0066] Similarly, the terminal device can obtain the center point set of each rod-shaped object to be identified in the image to be identified in the same manner, and the details are not repeated here. Figure 6 The schematic diagram of the segmentation result is shown in FIG. 1 , and the schematic diagram of the center point set of each rod-shaped object in the image to be identified can be as shown in FIG. Fig.10 shown.

[0067] 304. Obtain a first center line corresponding to the first rod-shaped object to be identified according to the center point in the first center point set.

[0068] After obtaining the center point set of each rod-shaped object to be identified, the terminal device determines the center line corresponding to the first rod-shaped object to be identified according to the center point set. The following takes the first rod-shaped object to be identified as an example for explanation, and the specific operation can be as follows: determine the horizontal coordinate variation range corresponding to the first center point set according to the horizontal coordinate of each center point in the first center point set; determine the specific method of determining the center line according to the center point set according to the horizontal coordinate variation range.

[0069] In a possible implementation, when the horizontal coordinate variation range is less than or equal to the first preset threshold, the first center line is defined as a vertical center line; wherein the horizontal coordinate of the first center line is the average of the horizontal coordinates of each center point in the first center point set, and the vertical coordinate of the first center line is the minimum vertical coordinate and the maximum vertical coordinate of each center point in the first center point set. In this embodiment, the first preset threshold can be set to 3. That is, when the horizontal coordinate variation range of each center point in the first center point set to be identified does not exceed 3, such as the horizontal coordinate variation range of each center point in the first center point set is (36, 38), the center points of the first to-be-identified rod-shaped object are close to being arranged vertically, and the center line is likely to be a vertical straight line. Since the least squares method cannot be used to fit a vertical straight line, the center line can be directly defined as a vertical center line x=b, wherein b is the average of the horizontal coordinates of each center point in the first to-be-identified rod-shaped object center point set.

[0070] In a possible implementation, when the horizontal coordinate variation range is greater than the first preset threshold and less than the second preset threshold, each center point of the first center point set is fitted to obtain the first center line. In this embodiment, the first preset threshold can be set to 3, and the second preset threshold can be set to 10. That is, when the horizontal coordinate variation range of each center point in the first center point set of the rod-shaped object to be identified exceeds 3 but does not exceed 10, each center point of the rod-shaped object to be identified can be fitted by the least square method to obtain the center line of the first rod-shaped object to be identified. For example, if the horizontal coordinate variation range of the center point in the first center point set is (36, 41), the terminal device uses the least square method to fit each center point of the rod-shaped object to be identified to obtain the center line of the first rod-shaped object to be identified.

[0071] When the range of the horizontal coordinate change is greater than or equal to the second preset threshold, the abnormal center point is discarded from the first center point set to obtain the second center point set, and the center points of the second center point set are fitted to obtain the first center line, and the abnormal center point is the center point whose horizontal coordinate and the average of the horizontal coordinates of each center point in the first center point set exceed the third preset threshold. In this embodiment, the second preset threshold can be set to 10. That is, when the range of the horizontal coordinate of each center point in the center point set of the first rod-shaped object to be identified exceeds 10, the terminal device determines that the first center point set includes an abnormal center point, and then the terminal device calculates the average of the horizontal coordinates of each center point in the first center point set, and calculates the difference between the horizontal coordinates of each center point and the average. If the difference exceeds the third preset threshold, the terminal device determines that the center point is an abnormal center point. Then the terminal device discards the abnormal center point and uses the least squares method to fit the remaining center points to obtain the center line of the rod-shaped object to be identified. For example, the horizontal coordinate variation range of the center point in the first center point set is (36, 50), and the terminal device determines that the center point with a horizontal coordinate of 50 is an abnormal center point, then the center point with a horizontal coordinate of 50 is discarded, and then the least squares method is used to fit the remaining center points of the rod-shaped object to be identified to obtain the midline of the first rod-shaped object to be identified.

[0072] In this embodiment, combined with Fig.10 As shown, the center line of each rod-shaped object to be identified in the image to be identified can be as follows: Fig.11 shown.

[0073] 305. Determine position information of the first rod-shaped object to be identified according to the first center line.

[0074] After determining the center lines of the rod-shaped objects to be identified, the terminal device may determine the position information of the first rod-shaped object to be identified in the image to be identified.

[0075] In this embodiment, after determining the position information of the rod-shaped object to be identified, the terminal device can compare the position relationship between the vehicle and the rod-shaped object to be identified with the data stored in the high-precision map during the vehicle's driving process to deduce the precise vehicle position.

[0076] The rod-shaped object recognition device in this application is described in detail below. Fig.12 , Fig.12 This is a schematic diagram of an embodiment of a rod-shaped object recognition device in an embodiment of the present application. The rod-shaped object recognition device 20 includes:

[0077] An acquisition module 201 is used to acquire an image to be recognized, the image to be recognized includes at least one rod-shaped object to be recognized; the image to be recognized is input into a segmentation network to obtain a first segmentation result, the first segmentation result is a segmentation result of the first rod-shaped object to be recognized in the image to be recognized, and the segmentation network is an image segmentation network based on semantic segmentation obtained by training the training image data and the initial segmentation network;

[0078] The processing module 202 is used to scan the first segmentation result to obtain a first center point set corresponding to the first rod-shaped object to be identified; and obtain a first midline corresponding to the first rod-shaped object to be identified according to the center point in the first center point set;

[0079] The determination module 203 is used to determine the position information of the first rod-shaped object to be identified according to the first center line.

[0080] In an embodiment of the present application, a rod-shaped object recognition device is provided. With the above device, after obtaining a prediction result based on semantic segmentation of the rod-shaped object in the image to be recognized, a set of center points of the rod-shaped object is calculated based on the segmentation result, and then the midline of the rod-shaped object is obtained, and finally the position information of the rod-shaped object in the image to be recognized is determined based on the midline, thereby obtaining the precise position information of the rod-shaped object.

[0081] Optionally, in the above Fig.12 On the basis of the corresponding embodiment, in another embodiment of the rod-shaped object identification device 20 provided by the embodiment of the present application, the first segmentation result is a pixel-level annotation, and the processing module 202 is specifically used to perform pixel scanning on the first segmentation result row by row to determine the first horizontal coordinate and the second horizontal coordinate of the first segmentation result on each row, wherein the first horizontal coordinate is the horizontal coordinate of the first appearance position of the first segmentation result in each row, and the second horizontal coordinate is the horizontal coordinate of the last appearance position of the first segmentation result in each row; the middle value of the first horizontal coordinate and the second horizontal coordinate is taken to obtain the horizontal coordinate of the first center point of the first segmentation result in each row, and then the vertical coordinate of each row is taken as the vertical coordinate of the first center point, and the first center point set is generated according to the first center point.

[0082] In an embodiment of the present application, a rod-shaped object recognition device is provided. The above device is used to scan pixels line by line for the independent block mask, and obtain the horizontal coordinate of the first appearance position and the horizontal coordinate of the last appearance position of the mask on each line, and take the middle value of the two horizontal coordinates to obtain the horizontal coordinate of the center point of the mask on each line, and then obtain the vertical coordinate of the center point to obtain the position information of the center point. In this way, a large number of center points can be provided for determining the center line of the first rod-shaped object to be identified, so that the center line of the first rod-shaped object to be identified is more accurate.

[0083] Optionally, in the above Fig.12 On the basis of the corresponding embodiment, in another embodiment of the rod-shaped object identification device 20 provided in the embodiment of the present application, the first segmentation result is a pixel-level annotation, and the processing module 202 is specifically used to segment the first segmentation result according to a preset number of rows to obtain a scan line; perform pixel scanning on the scan line to determine a third horizontal coordinate and a fourth horizontal coordinate of the first segmentation result on the scan line, wherein the third horizontal coordinate is the horizontal coordinate of the first appearance position of the first segmentation result on the scan line, and the fourth horizontal coordinate is the horizontal coordinate of the last appearance position of the first segmentation result on the scan line; take the middle value of the third horizontal coordinate and the fourth horizontal coordinate to obtain the horizontal coordinate of the second center point of the first segmentation result on the scan line, and then take the vertical coordinate of the scan line as the vertical coordinate of the second center point, and generate the first center point set according to the second center point.

[0084] In an embodiment of the present application, a rod-shaped object recognition device is provided. With the above device, a scan line is taken according to a preset line for an independent block of masks, and then the scan line is pixel-scanned, and the horizontal coordinate of the first and last appearance positions of the mask on the scan line are obtained, and the middle value of the two horizontal coordinates is taken to obtain the horizontal coordinate of the center point of the mask on the scan line, and then the vertical coordinate of the center point is obtained to obtain the position information of the center point. In this way, the efficiency of rod-shaped object recognition can be increased.

[0085] Optionally, in the above Fig.12 On the basis of the corresponding embodiment, in another embodiment of the rod-shaped object identification device 20 provided in the embodiment of the present application, the processing module 202 is specifically used to determine the horizontal coordinate variation range corresponding to the first center point set according to the horizontal coordinate of each center point in the first center point set;

[0086] When the horizontal coordinate variation range is less than or equal to the first preset threshold, the first center line is defined as a vertical center line; wherein the horizontal coordinate of the first center line is the average of the horizontal coordinates of each center point in the first center point set, and the vertical coordinate of the first center line is the minimum vertical coordinate and the maximum vertical coordinate of each center point in the first center point set;

[0087] When the horizontal coordinate variation range is greater than the first preset threshold value and less than the second preset threshold value, performing fitting processing on each center point of the first center point set to obtain the first center line;

[0088] When the horizontal coordinate variation range is greater than or equal to the second preset threshold, the abnormal center point is discarded from the first center point set to obtain a second center point set, and the center points of the second center point set are fitted to obtain the first center line, and the abnormal center point is a center point whose horizontal coordinate difference with the average of the horizontal coordinates of each center point in the first center point set exceeds the third preset threshold.

[0089] In an embodiment of the present application, a rod-shaped object recognition device is provided. By using the above device, the center line determination method under different circumstances is performed according to the horizontal coordinate of each center point in the center point set, which can effectively ensure the accuracy of the center line result.

[0090] Optionally, in the above Fig.12 On the basis of the corresponding embodiment, in another embodiment of the rod-shaped object identification device 20 provided in the embodiment of the present application, as Fig.12a As shown, the acquisition module 201 is also used to acquire the training image data and the initial segmentation network, the training image data is annotated with rod-shaped objects, the initial segmentation network adopts an encoder-decoder structure, and the initial segmentation network includes a context information enhancement module for acquiring vertical information of the rod-shaped objects;

[0091] The rod-shaped object recognition device 20 further includes a training module 204, and the training module 204 is used to train the initial segmentation network according to the training image data to obtain the segmentation network.

[0092] In an embodiment of the present application, a rod-shaped object recognition device is provided. Using the above device, a context information enhancement module for obtaining vertical information of the rod-shaped object is designed in the segmentation network to increase feature extraction of the rod-shaped object, thereby improving the rod-shaped object recognition accuracy.

[0093] Optionally, in the above Fig.12 On the basis of the corresponding embodiment, in another embodiment of the rod-shaped object recognition device 20 provided in the embodiment of the present application, the context information enhancement module is a plurality of vertical asymmetric convolution layers of different scales, and the convolution kernel of the vertical asymmetric convolution layer is designed to be N*1, where N is a positive integer greater than 1; the convolution module of the encoder is a lightweight depthwise-pointwise convolution module.

[0094] In an embodiment of the present application, a rod-shaped object recognition device is provided. Using the above device, the context information enhancement module is designed as a plurality of vertical asymmetric convolution layers of different scales, so that the characteristic information of the rod-shaped object can be more effectively extracted. At the same time, the segmentation network is designed as a lightweight depthwise-pointwise convolution module, which can reduce the computational requirements of the rod-shaped object recognition device.

[0095] Optionally, in the above Fig.12 On the basis of the corresponding embodiment, in another embodiment of the rod-shaped object recognition device 20 provided in the embodiment of the present application, the acquisition module 201 is specifically used to output the image to be recognized to the segmentation network to output M second segmentation results, and the M second segmentation results are the segmentation results of all the rod-shaped objects to be recognized in the image to be recognized, and M is an integer greater than or equal to 1;

[0096] An opening operation is performed on the M second segmentation results so that adhesions existing in the M second segmentation results are disconnected to obtain X independent block segmentation results, and the first segmentation result of the first rod-shaped object to be identified is included in the X independent block segmentation results, where X is an integer greater than or equal to M.

[0097] In an embodiment of the present application, a rod-shaped object recognition device is provided. With the above device, when there are multiple rod-shaped objects to be recognized in the image to be recognized, the output segmentation result needs to be adjusted so that the segmentation result is guaranteed to be an independent block mask, which can reduce the interference in the recognition process of different rod-shaped objects, thereby improving the accuracy of rod-shaped object recognition.

[0098] The rod-shaped object recognition device provided in this application may be a server. Fig.13 , Fig.13 : is a schematic diagram of a server structure provided in an embodiment of the present application. The server 300 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 322 (for example, one or more processors) and memory 332, and one or more storage media 330 (for example, one or more mass storage devices) storing application programs 342 or data 344. Among them, the memory 332 and the storage medium 330 can be short-term storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 322 can be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the server 300.

[0099] The server 300 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input and output interfaces 358, and / or one or more operating systems 341, such as Windows Server 2000. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0100] The steps performed by the rod-shaped object recognition device in the above embodiment can be based on the Fig.13 The server structure shown.

[0101] The rod-shaped object recognition device provided in this application can be a terminal device. Fig.14For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. In the embodiments of the present application, the terminal device is taken as an in-vehicle terminal as an example for explanation:

[0102] Fig.14 FIG. 1 is a block diagram showing a partial structure of a vehicle-mounted terminal related to a terminal device provided in an embodiment of the present application. Fig.14 The vehicle terminal includes: a radio frequency (RF) circuit 410, a memory 420, an input unit 430, a display unit 440, a sensor 450, an audio circuit 460, a wireless fidelity (WiFi) module 470, a processor 480, and a power supply 490. Those skilled in the art will appreciate that Fig.14 The vehicle-mounted terminal structure shown in the figure does not constitute a limitation on the vehicle-mounted terminal, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0103] Combine the following Fig.14 The following is a detailed introduction to the various components of the vehicle terminal:

[0104] The RF circuit 410 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 480 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 410 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 410 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0105] The memory 420 can be used to store software programs and modules. The processor 480 executes various functional applications and data processing of the vehicle terminal by running the software programs and modules stored in the memory 420. The memory 420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the vehicle terminal (such as audio data, a phone book, etc.), etc. In addition, the memory 420 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0106] The input unit 430 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the vehicle terminal. Specifically, the input unit 430 may include a touch panel 431 and other input devices 432. The touch panel 431, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 431 or near the touch panel 431 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 431 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 480, and can receive and execute the command sent by the processor 480. In addition, the touch panel 431 can be implemented in various types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch panel 431, the input unit 430 may also include other input devices 432. Specifically, other input devices 432 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0107] The display unit 440 can be used to display information input by the user or information provided to the user and various menus of the vehicle terminal. The display unit 440 may include a display panel 441. Optionally, the display panel 441 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 431 may cover the display panel 441. When the touch panel 431 detects a touch operation on or near it, it is transmitted to the processor 480 to determine the type of touch event. Subsequently, the processor 480 provides a corresponding visual output on the display panel 441 according to the type of touch event. Although in Fig.14 In the embodiment, the touch panel 431 and the display panel 441 are used as two independent components to realize the input and output functions of the vehicle terminal, but in some embodiments, the touch panel 431 and the display panel 441 can be integrated to realize the input and output functions of the vehicle terminal.

[0108] The vehicle terminal may also include at least one sensor 450, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 441 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 441 and / or the backlight when the vehicle terminal is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the vehicle terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; As for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can be configured in the vehicle terminal, they will not be repeated here.

[0109] The audio circuit 460, the speaker 461, and the microphone 462 can provide an audio interface between the user and the vehicle-mounted terminal. The audio circuit 460 can transmit the received audio data to the speaker 461 after converting the electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 462 converts the collected sound signal into an electrical signal, which is received by the audio circuit 460 and converted into audio data, and then the audio data is output to the processor 480 for processing, and then sent to another vehicle-mounted terminal through the RF circuit 410, or the audio data is output to the memory 420 for further processing.

[0110] WiFi is a short-range wireless transmission technology. The vehicle terminal can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 470. It provides users with wireless broadband Internet access. Fig.14 A WiFi module 470 is shown, but it is understandable that it is not an essential component of the vehicle-mounted terminal and can be omitted as needed without changing the essence of the invention.

[0111] The processor 480 is the control center of the vehicle terminal. It uses various interfaces and lines to connect various parts of the entire vehicle terminal. It executes various functions of the vehicle terminal and processes data by running or executing software programs and / or modules stored in the memory 420, and calling data stored in the memory 420, so as to monitor the vehicle terminal as a whole. Optionally, the processor 480 may include one or more processing units; optionally, the processor 480 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 480.

[0112] The vehicle terminal also includes a power supply 490 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 480 through a power management system, thereby managing functions such as charging, discharging, and power consumption management through the power management system.

[0113] Although not shown, the vehicle-mounted terminal may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0114] The steps performed by the terminal device in the above embodiment can be based on the Fig.14 The terminal device structure shown.

[0115] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the methods described in the aforementioned embodiments.

[0116] The present application also provides a computer program product including a program, which, when executed on a computer, enables the computer to execute the methods described in the aforementioned embodiments.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0118] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0119] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0120] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0121] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0122] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A rod-shaped object recognition method, characterized in that: include: Acquire an image to be identified, wherein the image to be identified includes at least one rod-shaped object to be identified; The rod-shaped object to be identified is a vertical pole in a traffic scene; Inputting the image to be identified into a segmentation network to obtain a first segmentation result, wherein the first segmentation result is a segmentation result of a first rod-shaped object to be identified in the image to be identified, and the segmentation network is an image segmentation network based on semantic segmentation obtained by training the training image data and the initial segmentation network; Performing pixel scanning on the first segmentation result to obtain a first center point set corresponding to the first rod-shaped object to be identified, wherein the first center point set includes center points of the first rod-shaped object to be identified obtained by scanning; Obtaining a first midline corresponding to the first rod-shaped object to be identified according to the central point in the first central point set; Determine the position information of the first rod-shaped object to be identified according to the first midline; The first segmentation result of the first rod-shaped object to be identified output by inputting the image to be identified into the segmentation network comprises: Output the image to be identified to the segmentation network to output M second segmentation results, where the M second segmentation results are segmentation results of all rod-shaped objects to be identified in the image to be identified, and M is an integer greater than or equal to 1; An opening operation is performed on the M second segmentation results so that adhesions in the M second segmentation results are disconnected to obtain X independent block segmentation results, and the first segmentation result of the first rod-shaped object to be identified is included in the X independent block segmentation results, where X is an integer greater than or equal to M.

2. The method according to claim 1, characterized in that The first segmentation result is represented by a pixel-level mask, and the pixel scanning of the first segmentation result to obtain a first set of center points corresponding to the first rod-shaped object to be identified includes: Scanning the first segmentation result row by row and determining a first horizontal coordinate and a second horizontal coordinate of the first segmentation result on each row, wherein the first horizontal coordinate is the horizontal coordinate of the first appearance position of the first segmentation result in each row, and the second horizontal coordinate is the horizontal coordinate of the last appearance position of the first segmentation result in each row; The middle value of the first horizontal coordinate and the second horizontal coordinate is taken to obtain the horizontal coordinate of the first center point of each row of the first segmentation result, and then the vertical coordinate of each row is taken as the vertical coordinate of the first center point, and the first center point set is generated according to the first center point.

3. The method according to claim 1, characterized in that The first segmentation result is represented by a pixel-level mask, and the pixel scanning of the first segmentation result to obtain a first set of center points corresponding to the first rod-shaped object to be identified includes: Segmenting the first segmentation result according to a preset number of lines to obtain scanning lines; Scanning the scan line by pixels to determine a third horizontal coordinate and a fourth horizontal coordinate of the first segmentation result on the scan line, wherein the third horizontal coordinate is the horizontal coordinate of a position where the first segmentation result appears for the first time on the scan line, and the fourth horizontal coordinate is the horizontal coordinate of a position where the first segmentation result appears for the last time on the scan line; The middle value of the third horizontal coordinate and the fourth horizontal coordinate is taken to obtain the horizontal coordinate of the second center point of the first segmentation result in the scanning line, and then the vertical coordinate of the scanning line is taken as the vertical coordinate of the second center point, and the first center point set is generated according to the second center point.

4. The method according to claim 1, characterized in that: The step of obtaining a first midline corresponding to the first to-be-identified rod-shaped object from a central point in the first central point set comprises: Determine a horizontal coordinate variation range corresponding to the first set of center points according to the horizontal coordinates of each center point in the first set of center points; When the horizontal coordinate variation range is less than or equal to the first preset threshold, the first center line is defined as a vertical center line; wherein the horizontal coordinate of the first center line is the average of the horizontal coordinates of each center point in the first center point set, and the vertical coordinate of the first center line is the minimum vertical coordinate and the maximum vertical coordinate of each center point in the first center point set; When the horizontal coordinate variation range is greater than the first preset threshold value and less than the second preset threshold value, performing fitting processing on each center point of the first center point set to obtain the first center line; When the horizontal coordinate variation range is greater than or equal to the second preset threshold, abnormal center points are discarded from the first center point set to obtain a second center point set, and the center points of the second center point set are fitted to obtain the first center line, and the abnormal center point is a center point whose horizontal coordinate difference with the average of the horizontal coordinates of each center point in the first center point set exceeds a third preset threshold.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquire the training image data and the initial segmentation network, wherein rods are annotated in the training image data, the initial segmentation network adopts an encoder-decoder structure, and the initial segmentation network includes a context information enhancement module for acquiring vertical information of the rods; The initial segmentation network is trained according to the training image data to obtain the segmentation network.

6. The method according to claim 5, characterized in that The context information enhancement module is a plurality of vertical asymmetric convolution layers of different scales, and the convolution kernel of the vertical asymmetric convolution layer is designed to be N*1; the convolution module of the encoder is a lightweight depthwise-pointwise convolution module, and N is a positive integer greater than 1.

7. A rod-shaped object recognition device, characterized in that: include: An acquisition module, used for acquiring an image to be identified, wherein the image to be identified includes at least one rod-shaped object to be identified; Inputting the image to be identified into a segmentation network to obtain a first segmentation result, wherein the first segmentation result is a segmentation result of a first rod-shaped object to be identified in the image to be identified, and the segmentation network is an image segmentation network based on semantic segmentation obtained by training the training image data and the initial segmentation network; the rod-shaped object to be identified is a vertical pole in a traffic scene; a processing module, configured to perform pixel scanning on the first segmentation result to obtain a first center point set corresponding to the first rod-shaped object to be identified, wherein the first center point set includes center points of the first rod-shaped object to be identified obtained by scanning; and obtain a first midline corresponding to the first rod-shaped object to be identified according to the center points in the first center point set; a determination module, configured to determine position information of the first rod-shaped object to be identified according to the first center line; The acquisition module is specifically used to output the image to be identified to the segmentation network to output M second segmentation results, wherein the M second segmentation results are segmentation results of all rod-shaped objects to be identified in the image to be identified, and M is an integer greater than or equal to 1; An opening operation is performed on the M second segmentation results so that adhesions in the M second segmentation results are disconnected to obtain X independent block segmentation results, and the first segmentation result of the first rod-shaped object to be identified is included in the X independent block segmentation results, where X is an integer greater than or equal to M.

8. The device according to claim 7, characterized in that The first segmentation result is a pixel-level annotation, and the processing module is specifically used to: Scanning the first segmentation result row by row and determining a first horizontal coordinate and a second horizontal coordinate of the first segmentation result on each row, wherein the first horizontal coordinate is the horizontal coordinate of the first appearance position of the first segmentation result in each row, and the second horizontal coordinate is the horizontal coordinate of the last appearance position of the first segmentation result in each row; The middle value of the first horizontal coordinate and the second horizontal coordinate is taken to obtain the horizontal coordinate of the first center point of each row of the first segmentation result, and then the vertical coordinate of each row is taken as the vertical coordinate of the first center point, and the first center point set is generated according to the first center point.

9. The device according to claim 7, characterized in that The first segmentation result is a pixel-level annotation, and the processing module is specifically used to: Segmenting the first segmentation result according to a preset number of lines to obtain scanning lines; Scanning the scan line by pixels to determine a third horizontal coordinate and a fourth horizontal coordinate of the first segmentation result on the scan line, wherein the third horizontal coordinate is the horizontal coordinate of a position where the first segmentation result appears for the first time on the scan line, and the fourth horizontal coordinate is the horizontal coordinate of a position where the first segmentation result appears for the last time on the scan line; The middle value of the third horizontal coordinate and the fourth horizontal coordinate is taken to obtain the horizontal coordinate of the second center point of the first segmentation result in the scanning line, and then the vertical coordinate of the scanning line is taken as the vertical coordinate of the second center point, and the first center point set is generated according to the second center point.

10. The device according to claim 7, characterized in that The processing module is specifically used for: Determine a horizontal coordinate variation range corresponding to the first set of center points according to the horizontal coordinates of each center point in the first set of center points; When the horizontal coordinate variation range is less than or equal to the first preset threshold, the first center line is defined as a vertical center line; wherein the horizontal coordinate of the first center line is the average of the horizontal coordinates of each center point in the first center point set, and the vertical coordinate of the first center line is the minimum vertical coordinate and the maximum vertical coordinate of each center point in the first center point set; When the horizontal coordinate variation range is greater than the first preset threshold value and less than the second preset threshold value, performing fitting processing on each center point of the first center point set to obtain the first center line; When the horizontal coordinate variation range is greater than or equal to the second preset threshold, abnormal center points are discarded from the first center point set to obtain a second center point set, and the center points of the second center point set are fitted to obtain the first center line, and the abnormal center point is a center point whose horizontal coordinate difference with the average of the horizontal coordinates of each center point in the first center point set exceeds a third preset threshold.

11. The device according to any one of claims 7 to 10, characterized in that The acquisition module is further used for: Acquire the training image data and the initial segmentation network, wherein rods are annotated in the training image data, the initial segmentation network adopts an encoder-decoder structure, and the initial segmentation network includes a context information enhancement module for acquiring vertical information of the rods; The rod-shaped object recognition device further includes a training module, wherein the training module is used to train the initial segmentation network according to the training image data to obtain the segmentation network.

12. The device according to claim 11, characterized in that The context information enhancement module is a plurality of vertical asymmetric convolution layers of different scales, and the convolution kernel of the vertical asymmetric convolution layer is designed to be N*1; the convolution module of the encoder is a lightweight depthwise-pointwise convolution module, and N is a positive integer greater than 1.

13. A computer device, characterized in that: include: Memory, processor, and bus system; Wherein, the memory is used to store programs; The processor is used to execute the program in the memory, and the processor is used to execute the method according to any one of claims 1 to 6 according to the instructions in the program code; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

14. A computer-readable storage medium comprising instructions, which, when executed on a computer, causes the computer to perform the method according to any one of claims 1 to 6.

15. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer executes the method according to any one of claims 1 to 6.