Line target recognition method and device, vehicle and equipment

Abstract

The application relates to a linear target recognition method, device, vehicle and equipment. The method comprises the following steps: acquiring a perception image, wherein the perception image comprises a linear target to be recognized; performing discrete sampling on the perception image to obtain a sampling image comprising discrete sampling results; identifying a feature region of the sampling image through a convolutional neural network to obtain position information of a sampling point corresponding to the linear target; and determining a recognition result of the linear target according to the position information of the sampling point corresponding to the linear target. The application can solve the problem that the number of linear target outputs is limited by performing discrete sampling on the perception image. The linear target recognition method has small calculation amount and wide applicability, and can be conveniently extended to detection of other linear objects.

Description

Technical Field

[0001] This application relates to the field of autonomous driving perception and recognition technology, and in particular to a method, device, vehicle, and equipment for recognizing linear targets. Background Technology

[0002] Neural networks are tools for large-scale, multi-parameter optimization. Relying on massive amounts of training data, neural networks can learn hidden features that are difficult to summarize from the data, thus completing many complex tasks, such as face detection, image semantic segmentation, object detection, action tracking, and natural language translation. Neural networks have been widely applied in the field of artificial intelligence. In recent years, Convolutional Neural Networks (CNNs) have achieved rapid development in the field of autonomous driving. For autonomous driving systems, the perception system is equivalent to the eyes of the autonomous vehicle, quickly identifying dynamic and static obstacles and the surrounding road structure. With the rapid embedding of CNNs, the perception systems in current autonomous vehicles all use neural networks for road environment cognition. Compared with traditional methods, convolutional neural networks often have higher recognition accuracy and robustness. However, when applying the image recognition methods of general convolutional neural networks to recognize linear targets, problems such as inaccurate recognition and complex computational processes arise. Summary of the Invention

[0003] Therefore, it is necessary to provide a method, device, vehicle, or computer equipment for linear target recognition based on neural networks to address the aforementioned technical problems.

[0004] In a first aspect, this application provides a method for recognizing linear targets based on neural networks, the method comprising:

[0005] Acquire a perceptual image, wherein the perceptual image includes a linear target object;

[0006] Discrete sampling and processing of pixels in a perceptual image yields a perceptual image containing discrete sampling results, i.e., a sampled image.

[0007] The feature regions of the perceptual image containing discrete sampling results are identified by a convolutional neural network to obtain the location information of the sampling points corresponding to the linear target object;

[0008] The identification result is determined based on the location information of the sampling points.

[0009] In one embodiment, the convolutional neural network includes a backbone network and branch networks. The backbone network is used to identify and extract feature regions in the sampled image and output a feature image of a preset size to the branch networks.

[0010] In one embodiment, the branch network includes at least a first branch network and a second branch network, which generate feature images with different scales and / or generate feature images corresponding to different regions of the perceived image.

[0011] In one embodiment, the first branch network classifies the perceived image, perceives whether each sampling point in the feature region contains a linear target object, and perceives the location information of the sampling points containing the linear target object through the second branch network.

[0012] In one embodiment, after acquiring the perceived image, the method further includes: preprocessing the perceived image.

[0013] In one embodiment, the step of discretely sampling the perceived image to obtain a sampled image includes:

[0014] The pixels of the perceived image are sampled at preset intervals by a number of points;

[0015] A coordinate system is established based on the perceived image;

[0016] Obtain the coordinates of the sampling point;

[0017] Based on the coordinates of the sampling points, the sampled image corresponding to the perceived image is obtained.

[0018] In one embodiment, the offset value of the sampling point coordinates relative to the center point of the feature region is the position information of the sampling point.

[0019] In one embodiment, determining the recognition result of the linear target object based on the location information of the sampling points corresponding to the linear target object includes:

[0020] Based on the position information of the sampling points corresponding to the linear target, the offset of the linear target in the sampling direction is determined;

[0021] The identification result of the linear target is determined based on the offset of the linear target in the sampling direction.

[0022] In one embodiment, the coordinates of the sampling point are (x i y i If x i =si-1, where y is the input sampling point data of the neural network. i Value, where x i Let y be the x-coordinate of the sampling point. i Let be the ordinate of the sampling point, and s be the preset value of the pixel sampling interval of the perceived image, i = 0, 1, 2, 3...

[0023] Secondly, this application also provides a linear target recognition device based on a neural network, the device comprising:

[0024] The acquisition module is used to acquire a perceived image, which includes a linear target object to be identified.

[0025] The processing module is pre-configured with a convolutional neural network; the processing module performs discrete sampling on the pixels of the perceived image to obtain a sampled image containing discrete sampling results; the processing module identifies the sampled image to obtain the position information of the sampling points corresponding to the linear target; and determines the recognition result of the linear target based on the position information of the sampling points corresponding to the linear target.

[0026] The transmission module is used to transmit data and process the recognition results obtained by the processing module.

[0027] Thirdly, this application also provides a vehicle that, using the aforementioned identification device, further includes:

[0028] A control device is connected to the identification device. The control device calculates based on the results of the processing module and determines the passage status of the vehicle. If the determination is correct, the control device controls the vehicle to move forward.

[0029] In one embodiment, the control device performs the following steps:

[0030] Select or set a reference frame, fit the recognition results, and obtain the equation of the linear target object;

[0031] Based on the equation of the linear target object, the relative position of the linear target object and the reference frame is calculated, and the state of the target object is determined based on the relative position.

[0032] The state of the target object is used as the basis for determining whether the vehicle is moving forward.

[0033] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0034] Acquire a perceptual image, wherein the perceptual image includes a linear target object to be identified;

[0035] Discretely sample the pixels of the perceived image to obtain a sampled image containing the discrete sampling results;

[0036] The convolutional neural network identifies the feature regions of the sampled image to obtain the location information of the sampling points corresponding to the linear target object;

[0037] The identification result of the linear target is obtained based on the location information of the sampling points corresponding to the linear target.

[0038] In one embodiment, the convolutional neural network includes a backbone network and branch networks, wherein the branch networks include a first branch network and a second branch network; the step of identifying the sampled image through the convolutional neural network to obtain the location information of the sampling points corresponding to the linear target objects in the perceived image includes:

[0039] The backbone network extracts the feature regions in the sampled image, and the identified feature regions are transmitted to the first branch network and the second branch network respectively.

[0040] The first branch network senses whether each sampling point in the feature region contains a linear target object, and the second branch network senses the location information of the sampling points containing the linear target object.

[0041] In one embodiment, after acquiring the perceived image, the method further includes: preprocessing the perceived image.

[0042] In one embodiment, the step of discretely sampling the perceived image to obtain a sampled image includes:

[0043] The pixels of the perceived image are sampled at preset intervals;

[0044] A coordinate system is established based on the perceived image;

[0045] Obtain the coordinates of the sampling point;

[0046] Based on the coordinates of the sampling points, the sampled image corresponding to the perceived image is obtained.

[0047] In one embodiment, the offset value of the sampling point coordinates relative to the center point of the feature image is the position information of the sampling point.

[0048] In one embodiment, determining the recognition result of the linear target object based on the location information of the sampling points corresponding to the linear target object includes:

[0049] Based on the position information of the sampling points corresponding to the linear target, the offset of the linear target in the sampling direction is determined;

[0050] The identification result of the linear target is determined based on the offset of the linear target in the sampling direction.

[0051] In one embodiment, the coordinates of the sampling point are (x i y i If x i =si-1, where y is the input sampling point data of the neural network. i Value, where xi Let y be the x-coordinate of the sampling point. i Let be the ordinate of the sampling point, and s be the preset value of the pixel sampling interval of the perceived image, i = 0, 1, 2, 3...

[0052] The aforementioned linear target recognition methods, devices, vehicles, and equipment can solve the problem of limited output number of linear targets by discretely sampling the perceived image; the linear target recognition method of this application has low computational load, wide applicability, and can be easily extended to the detection of other linear objects. Attached Figure Description

[0053] Figure 1 This is a flowchart of a linear target recognition method in one embodiment;

[0054] Figure 2 This is a schematic diagram of the structure of a convolutional neural network in one embodiment;

[0055] Figure 3 This is a schematic diagram of a sampled image in one embodiment;

[0056] Figure 4 This is a schematic diagram of a neural network-based linear target recognition device in another embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] First, the application scenarios and some of the terms involved in the embodiments of this application will be explained.

[0059] The linear target recognition method, apparatus, and device provided in this application can be applied to image recognition application scenarios of driving vehicles, and are not limited to the field of fully autonomous driving technology. They should be considered to include similar image recognition fields such as assisted driving.

[0060] The linear target recognition method provided in this application can be implemented by a linear target recognition device, a computer device, a storage medium, or a computer program product device. Exemplarily, the device can be implemented through software and / or hardware.

[0061] The images of objects and / or perceived images containing linear objects involved in the embodiments of this application are mainly images of the gate arm of the turnstile. However, in other embodiments, they may include, but are not limited to, at least one of the following: images of utility poles, zebra crossings, or lane lines.

[0062] Deep learning is a technique that uses neural networks with at least two hidden layers to perform nonlinear transformations or representation learning on inputs, enabling various analytical activities through the construction of deep neural networks. Neural networks are categorized in many ways depending on the number of neurons, common types include feedforward neural networks (FNNs), convolutional neural networks (CNNs)—encoding spatial correlations, recurrent neural networks (RNNs)—encoding temporal correlations, and generative adversarial networks (GANs). In a broad sense, deep neural networks include any of the above and combinations thereof; in a narrow sense, deep neural networks refer specifically to feedforward neural networks. Simple feedforward neural networks may even consist of only an input and output layer, while more complex feedforward neural networks with hidden layers are also called multilayer perceptrons (MLPs) and typically have three parts: an input layer, an output layer, and at least one hidden layer. Compared to other image classification algorithms, CNNs use relatively little preprocessing. This means the network learns manually designed filters, a feature designed independently of traditional algorithms. This independence from prior knowledge and human intervention in feature design is a major advantage. CNNs have recognition applications in image and video recognition, as described in this paper, through detection and classification.

[0063] In scenarios involving autonomous vehicles passing through turnstiles, the key information is identifying whether the turnstile barrier is in a state that allows passage. Current technologies typically feed the aforementioned image into a neural network to directly train a classification network. However, due to the difficulty in distinguishing critical points, classification errors are frequently encountered. In some visual detection tasks using deep learning neural networks, rectangular bounding boxes are often used to represent a target.

[0064] In this embodiment, if the gate arm is represented by a rectangle, the width of the rectangle will be much greater than its height when the gate arm is closed due to the size ratio of the gate arm itself. Simultaneously, when the gate arm is raised, the height of the rectangle is much greater than its width. Throughout the entire process of the gate arm changing from closed to raised, the aspect ratio of the rectangle changes significantly, which is detrimental to the feature learning of the neural network model.

[0065] To address this technical problem, this application provides a linear target recognition method that utilizes the inherent linear characteristics of the gate arm to abstract it into a linear target representation.

[0066] Therefore, one embodiment of this application provides a method for recognizing linear targets. Figure 1 This is a flowchart illustrating the above identification method. Figure 1 As shown, the linear target recognition method provided in this embodiment may include:

[0067] S100. Acquire a perception image, wherein the perception image includes a linear target object to be identified.

[0068] In this step, a perceived image is acquired using an image sensing device (e.g., a camera), and the perceived image includes an image of a linear target object. Furthermore, the acquisition process in this step can be either a real-time image acquired by the device, or a pre-acquired and stored image, and includes a process of retrieving the image from a storage module.

[0069] After acquiring the perceived image, it can be preprocessed before discrete sampling. Further, the preprocessing includes compressing or cropping the image to a preset initial image size WxH, where W represents the initial image width and H represents the initial image height. In one preprocessing method of this embodiment, the perceived image is compressed to an initial image size of 512x288 pixels (WxH). The perceived image can be an RGB format image, and this embodiment includes, but is not limited to, such images.

[0070] S200. Discretely sample the pixels of the perceived image to obtain a sampled image containing the discrete sampling results.

[0071] In this embodiment, the perceived image in step S200 can also represent the initial image after the above preprocessing. In other embodiments, the perceived image can also be processed directly without preprocessing, and step S200 can be executed directly.

[0072] Specifically, step S200 also includes the following steps:

[0073] S201. Sample the pixels of the perceived image or the initial image at intervals of s points, where s = 0, 1, 2, 3...; further, select one or both of the horizontal or vertical directions for interval sampling, where the horizontal direction is parallel to the image width W and the vertical direction is parallel to the image height H.

[0074] In this embodiment, the initial image size WxH is 512x288 pixels, and s is 4. Sampling is performed every four pixels in the image width direction.

[0075] S202. Establish a coordinate system based on the perceived image;

[0076] In this embodiment, a coordinate system is established to describe the coordinates of the sampling points, with the bottom edge of the sensing image as the x-axis, the left side of the sensing image as the y-axis, and the intersection of the bottom edge and the left side of the sensing image as the origin.

[0077] S203. Obtain the coordinates of the sampling points as discrete sampling results;

[0078] In this embodiment, the target object is the gate's barrier arm. The sampling point coordinates are represented as (x... i y i If x i=si-1, where y is the input sampling point data of the neural network. i Value, where x i Let y be the x-coordinate of the sampling point. i Let i be the ordinate of the sampling point, i = 1, 2, 3, ...

[0079] Specifically, this embodiment requires sampling each row of the perceived image or the initial image sequentially, obtaining a set of sampling point coordinates for each row. For example, if the sampling interval is s and the image size is 512x288 pixels, then a total of 128 sampling points containing the linear target are collected from the discrete sampling of the row containing the linear target in the perceived image or the initial image. The mathematical formula for the sampling points containing the linear target is as follows:

[0080] pole = [(x1, y1), ... (x i ,y i ),...(x 128 ,y 128 (1)

[0081] Furthermore, if the sampling interval is 4, substituting s = 4 into the equation gives x. i =4i-1, then the above mathematical formula (1) can be simplified to the following form:

[0082] pole=[(3,y1),...(4i-1,y i ),...(511,y 128 (2)

[0083] It should be noted that, as Figure 3 As shown, the vertical lines in the figure are sampling lines. Since the linear target (such as a gate arm) only occupies a section in the middle of all 128 sampling points, the vertical coordinate of the place where the linear target and the sampling line do not intersect is zero. The above formulas (1) and (2) only represent the set of sampling point coordinates for the sensing image or the initial image containing the linear target. Since this embodiment needs to sample each row of the sensing image or the initial image, multiple sets of sampling point sets corresponding to similar formulas (1) and (2) can be obtained. Moreover, the x-coordinate value of each sampling point in each sampling set is known, and the y-coordinate value is unknown. For example, the x-coordinates of each sampling point in each sampling set are 3,...,4i-1,...,511 in sequence.

[0084] S204. Based on the coordinates of the sampling points, determine the sampling image corresponding to the perceived image;

[0085] In this embodiment, based on the position coordinates of each sampling point in each set of samples collected in S203, the perceived image is divided into grids to obtain the sampled image corresponding to the perceived image. For example, Figure 3The diagram shown is a schematic of the sampled image obtained after discrete sampling of the perceived image. In this embodiment, the sampling result is specifically the coordinates of all discrete sampling points (x, y, z). i y i The coordinate matrix formed by the sampling points is then used. The coordinates of all the obtained sampling points are labeled in the image, that is, each perceived image or initial image corresponds to a label text, which contains the coordinate matrix of the sampling points. For sampling points that do not contain linear targets, the coordinates of the sampling results are known. For sampling points that contain linear targets, the sampling results are as shown in equation (2), which contains unknown y-coordinate information. In this embodiment, the value of the horizontal coordinate has been determined for each sampling point, so when using the convolutional neural network in subsequent steps, it is only necessary to predict the corresponding y. Furthermore, the offset of the sampling point coordinates relative to the center point of the feature image is specifically chosen as the position information of the sampling point.

[0086] S300: The sampling image is identified through a convolutional neural network to obtain the location information of the sampling points.

[0087] Convolutional neural networks (CNNs) mainly consist of several types of layers: input layers, convolutional layers, pooling layers, and fully connected layers. The input layer takes raw data or data preprocessed by other algorithms and feeds it into the CNN.

[0088] like Figure 2 As shown, the convolutional neural network 1 in this embodiment includes a backbone network 20 and branch networks 30. The branch networks 30 include a first branch network 31 and a second branch network 32. The backbone network 20 is used to identify feature regions in the sampled image and output a feature image of a preset size to the branch networks. Further, the backbone network 20 transmits the identified feature regions to the first branch network 31 and the second branch network 32 respectively. The backbone network 20 mainly consists of multiple convolutional layers and pooling layers. In this embodiment, there are at least two convolutional layers and at least two pooling layers. The convolutional layers of the convolutional neural network 1 extract features from the input data and abstract the implicit correlations in the original data through the convolutional kernel matrix. The convolutional neural network 1 typically inserts a pooling layer periodically between consecutive convolutional layers. The function of the pooling layer is to gradually reduce the spatial size of the data volume, thereby reducing the number of parameters in the network and reducing the consumption of computational resources.

[0089] The convolutional neural network identifies feature regions in a perceived image. Further, after the sampled image 10 is input into the backbone network 20 of the convolutional neural network 1, feature extraction is performed via the backbone network 20 to obtain an intermediate feature image 21 (i.e., the feature region). In this embodiment, as... Figure 2 As shown, the size of the intermediate feature image 21 output by the backbone network 20 is wxh = W / 32xH / 32 = 16x9 pixels.

[0090] The backbone network 20 is connected to at least two branch networks 30, including a first branch network 31 and a second branch network 32. The first branch network 31 and the second branch network 32 generate feature maps with different scales and / or generate feature maps corresponding to different regions of the perceived image. The first branch network 31 senses whether each sampling point in the feature region contains a linear target object, and senses the location information of the sampling points containing linear target objects through the second branch network 32. It should be noted that in this embodiment, "connection" can mean that the output of the previous network is used as the input of the next network in the direction of signal transmission. For example, "the backbone network 20 is connected to at least two branch networks 30" can mean that the output of the backbone network 20 is used as the common input of the two branch networks 30. The first branch network 31 classifies the intermediate feature image 21, specifically to determine whether each sampling point contains a linear target object, that is, to classify each sampling point into a category containing linear target objects or a category not containing linear target objects. The second branch network 32 supervises the first branch network 31. Furthermore, the first branch network 31 judges all intermediate feature images 21 based on training, determining the classification of each intermediate feature image 21 by calculating probability values. The first branch network 31 has two channels, outputting one class of sampling points containing linear targets and another class of sampling points not containing linear targets. In this embodiment, the linear target is a gate arm. Furthermore, the second branch network 32 is a regression network that continuously calculates the coordinate values ​​and difference values ​​of the data to determine the final recognition result. In this embodiment, the second branch network 32 has 128 channels, and each channel's intermediate feature image 21 contains corresponding sampling points. This second branch network 32 is used to output the location information of sampling points containing linear targets. Optionally, for sampling points not containing linear targets, their location information may not be output, or their location information may be defaulted to 0.

[0091] Before the convolutional neural network (CNN) performs actual predictions, it needs to be trained using a pre-defined training set. The training process of CNN1 includes: acquiring or inputting sampled images to construct a training set, where the sampled images include linear targets; inputting the training set into the CNN, and using a loss function to obtain the final neural network model. Specifically, when constructing the training set, it is necessary to sample the perceptual images and input the indexed sampled images into the neural network for learning. The specific sampling method is as described above. During training, a corresponding loss function needs to be constructed. The loss function designed for linear targets is as follows:

[0092]

[0093] In equation (3), the loss function of the first branch network 31 is loss.cls The loss function of the second branch network 32 offset The calculation formulas are as follows:

[0094] loss cls =-[c t lnc p +(1-c t )ln(1-c p (4)

[0095]

[0096] For the classification branch, the loss function used in this embodiment is the cross-entropy loss function; for the coordinate offset regression branch, the loss function used in this embodiment is the squared difference loss function. l obj The value is 1 if a barrier passes through the (i,j)th grid cell, and 0 otherwise; N cls This represents the size of the final feature map, which is the number of grid cells, N. cls =wxh.c t Let c represent the truth value in the (i,j)th grid. p Let y represent the predicted value in the (i,j)th grid. k p y represents the value of the neural network prediction offset. k gt This represents the true value of the offset. The training objective of the convolutional neural network to be trained is to minimize the system loss value. By continuously adjusting the parameters of the convolutional neural network based on the training results, the desired convolutional neural network can be constructed.

[0097] Furthermore, the backbone network 20 of the convolutional neural network 1 performs convolution and pooling on the sampled image to obtain an intermediate feature image 21 (i.e., a feature region); the branch network classifies and regresses the intermediate feature image 21 to obtain the location information of the sampling points corresponding to the linear target objects in the perceived image.

[0098] Optionally, in this embodiment, in this step, the location information of the sampling points corresponding to the linear target in the perceived image, after classification by the first branch network 31 and supervised calculation by the second branch network 32, can be c discrete sampling points y of the linear target (gate arm). ci Relative to the grid center point y center The offset Δy in the y-direction coordinate ci =y ci -y center .

[0099] S400. Based on the location information of the sampling points corresponding to the linear target, the recognition result is obtained.

[0100] In this embodiment, the identification result of the linear target object can be the position of the linear target object, the state of the linear target object, or the offset angle of the linear target object relative to the sampling direction, etc.

[0101] Furthermore, if the identification result of the linear target object is the state of the linear target object, then S400 includes the following steps:

[0102] S401. Select or set a reference frame; In this embodiment, the bottom edge and side edge of the perceived image are selected as the x-axis and y-axis of the coordinate system, respectively.

[0103] S402. Process the location information of the corresponding sampling points of the linear target object, and fit the equation of the linear target object;

[0104] In step S302, the convolutional neural network has obtained the position information of the sampling points corresponding to the linear target object. This information can be the x-axis and y-axis coordinates of the sampling points, or the offset of the y-axis, i.e., Δy. ci If the position information is an offset, then the coordinates of the sampling points corresponding to the linear target can be obtained through certain calculations, that is, the coordinates of the gate arm pixels in the feature region. Based on the position coordinates of the sampling points corresponding to the linear target, this embodiment uses the least squares method to perform linear fitting to obtain the equation of the gate arm. If the position information is the position coordinates of the sampling points corresponding to the linear target, then the equation of the linear target (such as a gate arm) can be obtained directly through fitting.

[0105] S403. Based on the equation of the linear target object, calculate the relative position of the linear target object and the reference frame, determine the state of the target object based on the relative position, and output the recognition result.

[0106] Furthermore, in this embodiment, based on the equation and coordinate system of the gate arm, the slope of the gate arm equation can be calculated, and then the tilt angle of the gate arm can be calculated. Based on the change in the tilt angle of the gate arm, it can be determined whether the gate can allow vehicles to pass, and the recognition result can be output.

[0107] Furthermore, if the identification result of the linear target object is the tilt angle of the linear target object, then S400 can also determine the offset of the linear target object in the sampling direction based on the position information of the sampling point corresponding to the linear target object; and determine the identification result of the linear target object based on the offset of the linear target object in the sampling direction. Specifically, the difference in coordinate values ​​between the sampling point corresponding to the linear target object and the center point of the sampling grid area where the sampling point is located in the sampling direction can be calculated as the offset of the linear target object in the sampling direction, and the tilt angle of the linear target object relative to a certain direction can be determined based on the offset of the sampling direction, and the tilt angle of the linear target object is taken as the final identification result of the linear target object. If the linear target object is a turnstile, the tilt angle of the turnstile relative to the horizontal direction can be determined as the identification result of the turnstile status.

[0108] This embodiment uses a linear target object as the gate arm as an example. By detecting the gate arm sampling points, the corresponding angle change during the lifting process of the gate arm is obtained. Finally, by threshold control, the state of whether the gate allows passage can be effectively obtained, making it convenient to monitor the gate arm status in real time. By measuring the angle change, the critical value of the gate arm from closed to raised to allow passage can be obtained more accurately, and the given gate status is more precise. The method of directly classifying images in this invention has strong robustness and can also additionally obtain the gate arm angle, making it convenient to inform the driver of the current gate arm status.

[0109] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0110] Based on the same inventive concept, embodiments of the present invention also provide a linear target recognition device for implementing the neural network-based linear target recognition method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the linear target recognition device provided below can be found in the limitations of the linear target recognition method described above, and will not be repeated here.

[0111] like Figure 4As shown, in this embodiment, the linear target recognition device 100 includes a data acquisition module 101, a data storage module 103, and an output module 104. The linear target recognition device 100 is mounted on the terminal 200. The terminal 102 communicates with the linear target recognition device 100 via the transmission module 104. The data storage module 103 can store or cache data that the processing module 102 needs to process. The data storage module 103 can be integrated into the processing module 102 or placed on a cloud or other server. The server can be a standalone server or a server cluster composed of multiple servers. The data acquisition module 101 is used to acquire a perceived image, which includes linear targets. Further, the data acquisition module can be a camera, etc., and the perceived image can be a picture or video. The processing module 102 is pre-loaded with the trained convolutional neural network described above. The processing module 102 performs discrete sampling and processing on the pixels of the perceived image; identifies the feature regions of the perceived image to obtain a feature image containing the sampling results; and obtains the recognition result based on the feature image. The processing module 102 transmits the recognition result to the terminal 200 via the transmission module 104. Terminal 200 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. In this embodiment, terminal 200 is a vehicle.

[0112] Each module in the aforementioned linear target recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0113] This invention also provides an application scenario for implementing the neural network-based linear target recognition method described above. This embodiment provides a vehicle equipped with a linear target recognition device 100. The vehicle also includes a control device connected to the linear target recognition device 100. The control device calculates based on the recognition result from the processing module 102 and determines the vehicle's passage status. If the determination is correct, the control device controls the vehicle to move forward. The control device performs the following steps: selecting or setting a reference frame; fitting the recognition result to obtain the target object equation; calculating the relative position of the target object and the reference frame, and determining the target object's state based on the relative position; using the target object's state as the basis for determining whether the vehicle should move forward. Further, in this embodiment, the control device performs step S400 of the neural network-based linear target recognition method described above. Based on the change in the gate arm's tilt angle, it determines whether the gate's state allows the vehicle to pass. The angle is compared with a preset threshold. If the requirements are met, the control device controls the vehicle to move forward or issues a forward signal. Furthermore, when the vehicle in this embodiment is a vehicle using autonomous driving technology or the vehicle is set to autonomous driving mode, the control device automatically controls the vehicle to move forward; when the vehicle in this embodiment is an ordinary vehicle, the control device gives a signal that it can move forward.

[0114] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores image and computational process data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the aforementioned neural network-based linear target recognition method.

[0115] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for recognizing linear targets, characterized in that, The method includes: Acquire a perception image, wherein the perception image includes a linear target object to be identified; Discrete sampling is performed on the perceived image to obtain a sampled image containing the discrete sampling results; The convolutional neural network identifies feature regions in the sampled image to obtain the location information of sampling points corresponding to linear targets. The convolutional neural network includes a backbone network and branch networks. The backbone network identifies and extracts feature regions in the sampled image and outputs feature images of a preset size to the branch networks. The branch networks include at least a first branch network and a second branch network. The first and second branch networks generate feature images of different scales, or generate feature images corresponding to different regions of the sampled image. The first branch network classifies the sampled image. The second branch network perceives the location information of sampling points containing linear targets. The identification result of the linear target is determined based on the location information of the sampling points corresponding to the linear target. The step of discretely sampling the perceived image to obtain a sampled image includes: sampling the pixels of the perceived image at preset intervals; establishing a coordinate system based on the perceived image; obtaining the coordinates of the sampling points; and obtaining the sampled image corresponding to the perceived image based on the coordinates of the sampling points; wherein the offset value of the coordinates of the sampling points relative to the center point of the feature region is the position information of the sampling points. The coordinates of the sampling points are (x) i y i ), then x i =si-1, where y is the input sampling point data of the neural network. i Value, where x i Let y be the x-coordinate of the sampling point. i Let be the ordinate of the sampling point, and si be the preset value of the pixel sampling interval of the perceived image, i = 0, 1, 2, 3...

2. The linear target recognition method according to claim 1, characterized in that, The backbone network includes at least two convolutional layers and at least two pooling layers.

3. The linear target recognition method according to claim 1, characterized in that, After acquiring the perceived image, the method further includes: preprocessing the perceived image.

4. The linear target recognition method according to any one of claims 1 to 3, characterized in that, The convolutional neural network is trained according to the following training process: Acquire or input sample images to construct a training set, wherein the sample images include linear target objects; The training set is input into the convolutional neural network, and the final convolutional neural network is obtained by using a loss function designed for linear targets.

5. The linear target recognition method according to claim 1, characterized in that, The identification result of the linear target object includes the state of the linear target object. Determining the identification result of the linear target object based on the location information of the sampling points corresponding to the linear target object includes: Select or set a reference frame; The position information of the corresponding sampling points of the linear target object is processed, and the equation of the linear target object is obtained by fitting. Based on the equation of the linear target object, the relative position of the linear target object and the reference frame is calculated, and the state of the target object is determined according to the relative position, and the recognition result is output.

6. The linear target recognition method according to claim 1, characterized in that, The identification result of the linear target object includes the tilt angle of the linear target object. Determining the identification result of the linear target object based on the position information of the sampling points corresponding to the linear target object includes: Based on the position information of the sampling points corresponding to the linear target, the offset of the linear target in the sampling direction is determined; The tilt angle of the linear target is determined based on the offset of the linear target in the sampling direction.

7. A linear target identification device, characterized in that, include: The acquisition module is used to acquire a perceived image, which includes a linear target object to be identified. The processing module is pre-configured with a convolutional neural network; the processing module performs discrete sampling on the pixels of the perceived image to obtain a sampled image containing discrete sampling results; the processing module identifies the sampled image to obtain the position information of the sampling points corresponding to the linear target; and determines the recognition result of the linear target based on the position information of the sampling points corresponding to the linear target. The convolutional neural network includes a backbone network and branch networks. The backbone network is used to identify and extract feature regions in the sampled image and output feature images of a preset size to the branch networks. The branch networks include at least a first branch network and a second branch network. The first branch network and the second branch network generate feature images of different scales, or generate feature images corresponding to different regions of the sampled image. The first branch network classifies the sampled image. The second branch network perceives the location information of sampling points containing linear targets. The step of discretely sampling the perceived image to obtain a sampled image includes: sampling the pixels of the perceived image at preset intervals; establishing a coordinate system based on the perceived image; obtaining the coordinates of the sampling points; and obtaining the sampled image corresponding to the perceived image based on the coordinates of the sampling points; wherein the offset of the coordinates of the sampling points relative to the center point of the feature region is the position information of the sampling points. The coordinates of the sampling points are (x) i y i ), then x i =si-1, where y is the input sampling point data of the neural network. i Value, where x i Let y be the x-coordinate of the sampling point. i Let be the ordinate of the sampling point, and si be the preset value of the pixel sampling interval of the perceived image, i = 0, 1, 2, 3...

8. A vehicle employing the identification device as described in claim 7, characterized in that, Also includes: A control device is connected to the identification device. The control device calculates based on the results of the processing module and determines the passage status of the vehicle. If the determination is correct, the control device controls the vehicle to move forward.

9. The vehicle according to claim 8, characterized in that, The control device performs the following steps: Select or set a reference frame, fit the recognition results, and obtain the equation of the linear target object; Based on the equation of the linear target object, the relative position of the linear target object and the reference frame is calculated, and the state of the target object is determined based on the relative position. The state of the target object is used as the basis for determining whether the vehicle is moving forward.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

Images