A method for identifying leftovers by unmanned patrol vehicles

Abstract

The present invention relates to a method for identifying leftovers by an unmanned patrol vehicle. The method comprises a cloud server, a camera, a laser radar sensor and a high-precision positioning sensor, wherein the laser radar sensor and the high-precision positioning sensor are connected to a vehicle-mounted computer, and the cloud server is connected to the camera for communication. The present invention improves the recovery rate of lost items and reduces the economic losses caused by lost items in daily life through the joint cooperation of the unmanned patrol vehicle and the cloud dispatching platform.

Description

Technical Field

[0001] The invention relates to the technical field of unmanned patrol vehicles, and in particular to a method for an unmanned patrol vehicle to identify leftover objects. Background Art

[0002] CN107908667A discloses a lost and found method and device, the invention content of which only publishes the lost item information on the Internet. Its defect is that people are required to find the lost item in advance and upload the item information to the Internet for comprehensive search.

[0003] With the development of unmanned driving technology in recent years, its application fields have gradually expanded. At present, when unmanned patrol cars are driving, they often need to transmit obstacles to the background, which relies entirely on manual identification. This is labor-intensive for the background staff and cannot maintain 24-hour online identification. Therefore, a reliable method needs to be studied with the technical support of unmanned driving. Summary of the invention

[0004] In view of the above deficiencies in the prior art, the present invention provides a method for identifying left-behind objects by an unmanned patrol vehicle, thereby improving the recovery rate of lost objects.

[0005] In order to achieve the above-mentioned purpose and other related purposes, the technical solution of the present invention is as follows:

[0006] A method for identifying leftovers by an unmanned patrol vehicle comprises a cloud server, a camera, a laser radar sensor and a high-precision positioning sensor, wherein the laser radar sensor and the high-precision positioning sensor are connected to a vehicle-mounted computer, and the cloud server is connected to the camera for communication. The method comprises the following steps:

[0007] A. Set a maximum duration T k , in T k During the time period, multiple frames of images are picked up from the camera video at N frames / second, Tk>5 seconds, 5>N>2, the first frame image and the last frame image are retained, and they are converted into single-channel images, Gaussian filtered, and smoothed images respectively;

[0008] B. The judgment of the remnants of different sizes is that the object appears in the sensitive area and remains still for a period of time. The pixels exceeding this time are cleared to update the motion history image.

[0009]

[0010] Among them, d represents the actual feature map, m represents the historical feature map, time represents the current time, and Tk represents the maximum duration;

[0011] C. Use the invariant moment to determine the feature vector of the image. By comparing the invariant moment features of the current grid attribute image and the historical grid attribute image of each frame and judging their differences, the changes in the sensitive area are characterized. The expression is:

[0012]

[0013] in, and They are the HU matrices of the first frame and the last frame respectively;

[0014] D. By comparing the difference between the background image and the current frame image, a threshold is set for the difference. If the difference is greater than the threshold, it is determined that there are relics on the street;

[0015] E. Input the current frame image into the convolutional neural network for recognition, and send the predicted recognition result to the cloud server.

[0016] In one embodiment of the present invention, in step E, the convolutional neural network learning recognition specifically includes the following steps:

[0017] E1. The convolution layer initially extracts features and extracts image features. The weights of the convolution kernels are learnable. The convolution operation can break through the limitations of traditional filters and extract the desired features according to the objective function. Parameter sharing reduces network parameters and improves training efficiency.

[0018] E2. The pooling layer extracts the main features, selects the one with the largest value in the area and retains it, and takes the maximum value for each pooling area;

[0019] E3. The fully connected layer aggregates the features of each part, expands the output of the convolution layer and the pooling layer into a one-dimensional form, and then connects a regression network or a classification network with the same structure as the ordinary network.

[0020] E4. Generate a classifier for predictive recognition.

[0021] In one embodiment of the present invention, in step D, the cloud server pushes the alarm to the backstage personnel APP, and the backstage personnel confirms and takes processing measures based on the image sent back by the camera.

[0022] In one embodiment of the present invention, in step E, the cloud server pushes the prediction and recognition results to the backstage personnel APP, and the backstage personnel confirm and take processing measures based on the image sent back by the camera.

[0023] The present invention has the following beneficial effects: The present invention adopts convolutional neural network learning and recognition algorithms, which can quickly identify static objects in the street and determine the types of lost objects. Through preliminary classification, the labor intensity of the staff is greatly reduced. In addition, the cooperation between the present invention and the cloud scheduling platform can greatly improve the recovery rate of lost objects and reduce the economic losses caused by lost objects in daily life. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the structure of an unmanned patrol vehicle;

[0025] Figure 2 Workflow diagram of unmanned patrol car identifying lost items;

[0026] Component number explanation: 1-on-board computer, 2-wireless communication module, 3-camera, 4-lidar sensor, 5-on-board touch screen, 6-combined navigation module, 7-ultrasonic sensor. DETAILED DESCRIPTION

[0027] The following is a description of the implementation of the present invention by means of specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification.

[0028] The technical solution of the present invention is as follows:

[0029] A method for unmanned patrol vehicle to identify leftovers, such as Figure 1 As shown, it includes a cloud server, a camera, an ultrasonic sensor, a laser radar sensor and a high-precision positioning sensor, the laser radar sensor and the high-precision positioning sensor are connected to the vehicle-mounted computer, the cloud server is connected to the camera for communication, and the method includes the following steps:

[0030] A. Set a maximum duration T k , in T k During the time period, multiple frames of images are picked up from the camera video at N frames / second, T k >5 seconds, 5>N>2, keep the first frame image and the last frame image, convert them into single-channel images, Gaussian filter, and smooth the images;

[0031] B. The judgment of the remnants of different sizes is that the object appears in the sensitive area and remains still for a period of time. The pixels exceeding this time are cleared to update the motion history image.

[0032]

[0033] Among them, d represents the current feature map, m represents the historical feature map, time represents the current time, and T k Indicates the maximum duration;

[0034] C. Use invariant moments to determine the eigenvector of the image:

[0035]

[0036]

[0037] Center distance:

[0038]

[0039] The normalized center distance is:

[0040] Where r = p + q2 + 2;

[0041] Normalization is to offset the impact of scale change on the center distance, so that the image remains unchanged before and after scaling. By comparing the invariant moment features of the real grid attribute image and the historical grid attribute image of each frame and judging their differences, the changes in the sensitive area are characterized. The expression is:

[0042]

[0043] in, and They are the HU matrices of the first frame and the last frame respectively;

[0044] D. By comparing the difference between the background image and the current frame image, a threshold is set for the difference. If the difference is greater than the threshold, it is determined that there are relics on the street;

[0045] E. Input the current frame image into the convolutional neural network for recognition, and send the predicted recognition result to the cloud server.

[0046] In addition, the present invention provides a computer-readable storage medium storing one or more programs, wherein the computer-readable storage medium stores one or more program instructions, and when the program instructions are executed by a processor, any one of the above methods is completed.

[0047] In one embodiment of the present invention, in step E, the following steps are specifically included:

[0048] E1. The convolution layer initially extracts features and extracts image features. The weights of the convolution kernels are learnable. The convolution operation can break through the limitations of traditional filters and extract the desired features according to the objective function. Parameter sharing reduces network parameters and improves training efficiency.

[0049] E2. The pooling layer extracts the main features, selects the one with the largest value in the area and retains it, and takes the maximum value for each pooling area;

[0050] E3. The fully connected layer aggregates the features of each part, expands the output of the convolution layer and the pooling layer into a one-dimensional form, and then connects a regression network or a classification network with the same structure as the ordinary network.

[0051] E4. Generate a classifier for predictive recognition.

[0052] like Figure 2 As shown, in one embodiment of the present invention, as Figure 2 As shown, in step D, the cloud server pushes the alarm to the backstage personnel APP, and the backstage personnel confirms and takes processing measures based on the image sent back by the camera.

[0053] In one embodiment of the present invention, Figure 2 As shown, in step E, the cloud server pushes the prediction and recognition results to the backstage personnel APP, and the backstage personnel confirm and take processing measures based on the image sent back by the camera.

[0054] In summary, the present invention effectively improves the recovery rate of lost items and reduces the economic losses caused by lost items in life.

[0055] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for identifying leftovers by an unmanned patrol vehicle, comprising a cloud server, a camera, a laser radar sensor and a high-precision positioning sensor, wherein the laser radar sensor and the high-precision positioning sensor are connected to a vehicle-mounted computer, and the cloud server is connected to the camera for communication, characterized in that The method comprises the following steps: A. Set a maximum duration T k , in T k During the time period, multiple frames of images are picked up from the camera video at N frames / second, T k >5 seconds, 5>N>2, keep the first frame image and the last frame image, convert them into single-channel images, Gaussian filter, and smooth the images; B. The judgment of the remnants of different sizes is that after the object appears in the sensitive area and remains still for a period of time, the pixels exceeding this time are cleared to update the motion history image; Among them, d represents the current feature map, m represents the historical feature map, time represents the current time, and T k Indicates the maximum duration; C. Use the invariant moment to determine the feature vector of the image. By comparing the invariant moment features of the current grid attribute image and the historical grid attribute image of each frame and judging their differences, the changes in the sensitive area are characterized. The expression is: in, and They are the HU matrices of the first frame and the last frame respectively; D. By comparing the difference between the background image and the current frame image, a threshold is set for the difference. If the difference is greater than the threshold, it is determined that there are relics on the street; E. Input the current frame image into the convolutional neural network for recognition, and send the predicted recognition result to the cloud server.

2. The method for identifying leftovers by an unmanned patrol vehicle according to claim 1, characterized in that: In step E, the following steps are specifically included: E1. The convolution layer initially extracts features and extracts image features. The weights of the convolution kernels are learnable. The convolution operation can break through the limitations of traditional filters and extract the desired features according to the objective function. E2. The pooling layer extracts the main features, selects the one with the largest value in the area and retains it, and takes the maximum value for each pooling area; E3. The fully connected layer aggregates the features of each part, expands the output of the convolution layer and the pooling layer into a one-dimensional form, and then connects a regression network or a classification network with the same structure as the ordinary network. E4. Generate a classifier for predictive recognition.

3. The method for identifying leftovers by an unmanned patrol vehicle according to claim 1, characterized in that: In step D, the cloud server pushes the alarm to the backstage personnel APP, and the backstage personnel confirms and takes processing measures based on the image sent back by the camera.

4. The method for identifying leftovers by an unmanned patrol vehicle according to claim 1, characterized in that: In step E, the cloud server pushes the prediction and recognition results to the backstage personnel APP, and the backstage personnel confirm and take processing measures based on the image sent back by the camera.

5. A computer-readable storage medium storing one or more programs, characterized in that: The computer-readable storage medium stores one or more program instructions, and when the program instructions are executed by a processor, any one of the methods in claims 1 to 4 is implemented.

Images