Target object detection method, device, electronic device and storage medium

Abstract

The present invention relates to the field of image detection and discloses a target object detection method, comprising: performing size standardization processing on a picture to be detected to obtain a standard picture, extracting picture features in the standard picture through a feature extraction network to obtain a feature picture; performing target detection on the feature picture using a region generation network to generate a candidate frame, and using a region feature aggregation algorithm to pool the candidate frame into a fixed size to obtain a standard candidate frame; regressing and classifying the standard candidate frame to obtain a target object candidate frame; performing coordinate mapping on the picture to be detected based on the target object candidate frame, and marking the target object detection result in the picture to be detected. The present invention also relates to blockchain technology, and the picture to be detected can be stored in a blockchain node. The present invention also proposes a target object detection device, an electronic device and a storage medium. The embodiments of the present invention solve the problem of inaccurate target detection results when the target object is blurred.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a target object detection method, device, electronic device and computer-readable storage medium. Background Art

[0002] Emergency lanes are dedicated lanes for vehicles handling emergency matters, such as engineering rescue, medical rescue, and police officers performing emergency duties. Traffic laws stipulate that motor vehicles may not occupy emergency lanes except under special circumstances, otherwise they will be penalized. Currently, deep learning methods based on regional convolutional neural networks can be used to detect emergency lanes. While these deep learning methods can achieve high accuracy in simple scenarios, their accuracy is relatively low in scenarios such as fog and haze, rainy days, at night, and when the emergency lane markings are blurred. Summary of the Invention

[0003] The present invention provides a target object detection method, device, electronic device and computer-readable storage medium, the main purpose of which is to improve the accuracy of target detection.

[0004] To achieve the above objectives, the present invention provides a target object detection method, comprising:

[0005] Obtain a picture to be detected, perform size normalization processing on the picture to be detected to obtain a standard picture, and extract picture features from the standard picture through a pre-built feature extraction network to obtain a feature picture;

[0006] Performing target detection on the feature image using a region generation network, generating candidate boxes based on the detection results, and pooling the candidate boxes into a fixed size using a region feature aggregation algorithm to obtain standard candidate boxes;

[0007] Regressing and classifying the standard candidate frame to obtain a target object candidate frame;

[0008] Coordinate mapping of the image to be detected is performed according to the target object candidate frame, and a target object detection result is marked in the image to be detected.

[0009] Optionally, performing size standardization on the image to be detected to obtain a standard image includes:

[0010] Determine whether the size of the image to be detected is larger than the standard image size input by the user;

[0011] When the size of the image to be detected is larger than the standard image size, cropping the image to be detected according to the standard image size to obtain a standard image;

[0012] When the size of the image to be detected is smaller than the standard image size, padding processing is performed on the image to be detected according to the standard image size to obtain a standard image.

[0013] Optionally, performing target detection on the feature image using a region generation network and generating a candidate box according to the detection result includes:

[0014] For each point on the feature image, generate a preset number of anchor boxes with different scales and aspect ratios;

[0015] Input the anchor box into the detection box classification layer of the region generation network for classification, and determine whether the feature map in the anchor box belongs to the foreground or the background;

[0016] Inputting the anchor box into the detection box regression layer of the region generation network to obtain the coordinate information of the anchor box;

[0017] An anchor box belonging to the foreground of the feature image is selected as a candidate box, and the candidate box is displayed on the feature image according to the corresponding coordinate value.

[0018] Optionally, the pooling of the candidate boxes into a fixed size using a regional feature aggregation algorithm to obtain a standard candidate box includes:

[0019] Divide each candidate box into n*n fixed-size units;

[0020] Determining sampling points in each of the units according to a preset rule, calculating pixel values ​​of the sampling points using a bilinear interpolation method, and performing a maximum pooling operation on the pixel values ​​of the sampling points to select a pixel point with the largest pixel value among the sampling points;

[0021] According to the selected pixel points, a standard candidate frame corresponding to each candidate frame is obtained.

[0022] Optionally, regressing and classifying the standard candidate frame to obtain the target object candidate frame includes:

[0023] Using a box regression function to obtain a predicted value of the offset of the standard candidate box relative to the actual position, so as to correct the standard candidate box;

[0024] The standard candidate frame is input into the fully connected layer and softmax function in the pre-trained neural network, the category to which the feature map in the standard candidate frame belongs is calculated, the score of the category is output, and the target detection frame is obtained according to the score.

[0025] Optionally, it is characterized in that, before extracting the image features from the standard image through the pre-built feature extraction network to obtain the feature image, the method further includes:

[0026] Construct the first convolution layer according to the convolution operation, normalization operation, and activation operation;

[0027] Use the merge function and the add function to construct the second convolution layer;

[0028] The feature extraction network is constructed according to the first convolutional layer and the second convolutional layer.

[0029] Optionally, the target object is an application lane.

[0030] In order to solve the above problems, the present invention further provides a target detection device, comprising:

[0031] An image feature extraction module is used to perform size standardization processing on the image to be detected to obtain a standard image, and to extract image features from the standard image through a pre-built feature extraction network to obtain a feature image;

[0032] A candidate box generation module is used to perform target detection on the feature image using a region generation network, generate candidate boxes based on the detection results, and pool the candidate boxes into a fixed size using a region feature aggregation algorithm to obtain a standard candidate box;

[0033] A classification and regression module is used to regress and classify the standard candidate frames to obtain target candidate frames;

[0034] The candidate box mapping module is used to perform coordinate mapping on the image to be detected according to the target candidate box, and mark the target detection result in the image to be detected.

[0035] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0036] at least one processor; and,

[0037] a memory communicatively connected to the at least one processor; wherein,

[0038] The memory stores computer program instructions that can be executed by the at least one processor. The computer program instructions are executed by the at least one processor to enable the at least one processor to perform the target object detection method described above.

[0039] In order to solve the above problems, the present invention also provides a computer-readable storage medium, including a data storage area and a program storage area, wherein the data storage area stores created data and the program storage area stores a computer program; wherein, when the computer program is executed by a processor, the above-mentioned target object detection method is implemented.

[0040] The embodiments of the present invention use a pre-built feature extraction network to extract features from the image to be detected, enhancing the ability to express features in difficult scenarios, thereby improving the overall accuracy of target detection, such as emergency lane detection. At the same time, the candidate frames generated by the region generation network and the region feature set algorithm are regressed and classified, effectively alleviating the pixel deviation problem and improving the regression positioning of the candidate frames for target detection, thereby further improving the overall accuracy of target detection. Therefore, the target detection method, device, and computer-readable storage medium proposed in the embodiments of the present invention can improve the accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a flow chart of a target object detection method provided by one embodiment of the present invention;

[0042] Figure 2 for Figure 1 A schematic diagram of a detailed implementation flow chart of one of the steps in the provided target object detection method;

[0043] Figure 3 for Figure 1 A schematic diagram of a detailed implementation flow chart of another step in the provided target object detection method;

[0044] Figure 4 for Figure 1 A schematic diagram of a detailed implementation flow chart of another step in a target object detection method is provided;

[0045] Figure 5 for Figure 1 A schematic diagram of a detailed implementation flow chart of another step in a target object detection method is provided;

[0046] Figure 6 A schematic diagram of a module of an object detection device provided by one embodiment of the present invention;

[0047] Figure 7 A schematic diagram of the internal structure of an electronic device for implementing a target object detection method provided by one embodiment of the present invention;

[0048] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0050] The present application provides a method for detecting an object. The object detection method may be performed by at least one electronic device, such as a server or a terminal, that can be configured to perform the method provided by the present application. In other words, the object detection method may be performed by software or hardware installed on a terminal device or a server device, where the software may be a blockchain platform. The server may include, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0051] Reference Figure 1 FIG. 1 is a flow chart of a target object detection method provided by an embodiment of the present invention. In this embodiment, the target object detection method includes:

[0052] S1. Obtain a picture to be detected, perform size standardization on the picture to be detected to obtain a standard picture, and extract picture features from the standard picture through a pre-built feature extraction network to obtain a feature picture.

[0053] In an embodiment of the present invention, various optical camera devices can be used to obtain the picture to be detected, such as using a traffic monitoring camera to photograph the emergency lane at specified time intervals, and uploading the photographed emergency lane pictures to a database, and finally summarizing the emergency lane pictures in the database to obtain the picture to be detected.

[0054] For details, see Figure 2 As shown, the step of performing size standardization on the image to be detected includes:

[0055] S10, determining whether the size of the image to be detected is larger than the standard image size input by the user;

[0056] S11. When the size of the image to be detected is larger than the standard image size, cropping the image to be detected according to the standard image size to obtain a standard image;

[0057] S12: When the size of the image to be detected is smaller than the standard image size, padding the image to be detected according to the standard image size to obtain a standard image.

[0058] For example, the standard image size can be set to 1000*600. If the size of the image to be tested is 1200*1200, the standard image can be obtained by center cropping with the center of the image as the origin, a length of 1000 and a width of 600. If the size of the image to be tested is 800*400, the image can be expanded outward with a preset pixel value using the image border as the boundary until the expanded size is 1000 in length and 600 in width, thus obtaining the standard image.

[0059] In an embodiment of the present invention, the feature extraction network may be a DarkNet63 network. Furthermore, in one embodiment of the present invention, the extracting of image features from the standard image by a pre-constructed feature extraction network also includes constructing the feature extraction network before obtaining the feature image.

[0060] In detail, the feature extraction network is constructed by the following method: constructing a first convolutional layer based on a convolution operation (Conv), a normalization operation (BN), and an activation operation (Leaky relu); combining the first convolutional layer using a merging function (Concat) and an adding function (Add) to construct a second convolutional layer; constructing the feature extraction network based on the first convolutional layer and the second convolutional layer.

[0061] Among them, the convolution operation is a 2D convolution operation, which is used to convolve the feature map from the standard image using 2D convolution kernels with different functions; the normalization operation is used to reduce the pixel values ​​of the pixels in the feature map using a normalization function; the activation operation uses an activation function to reduce the area size of the feature map; the merge function is used to connect two or more first convolution layers, and the add function is used to perform addition on the first convolution layer composition process.

[0062] S2. Use a region generation network to perform target detection on the feature image, generate a candidate frame based on the detection result, and use a region feature clustering algorithm to pool the candidate frame into a fixed size to obtain a standard candidate frame.

[0063] In an embodiment of the present invention, the region proposal network (RPN) is used to select candidate boxes from the feature image.

[0064] See Figure 3 As shown, the target detection is performed on the feature image using the region generation network, and a candidate frame is generated according to the detection result, including:

[0065] S20. For each point on the feature image, generate a preset number of anchor boxes with different scales and aspect ratios;

[0066] S21, inputting the anchor point frame into the detection and classification layer of the region generation network for classification, and determining whether the feature map in the anchor point frame belongs to the foreground or the background;

[0067] S22: Input the anchor box into the detection box regression layer of the region generation network to obtain the coordinate information of the anchor box;

[0068] S23. Select an anchor box belonging to the foreground of the feature map as a candidate box, and display the candidate box on the feature map according to the corresponding coordinate values.

[0069] In this embodiment of the present invention, for each point on the feature image, nine anchor boxes with different scales and aspect ratios can be generated. The nine anchor boxes are obtained from three different sizes and three different ratios. For example, the three sizes are 8, 16, and 32 (other sizes can also be set), and the three ratios are 1:1, 1:2, and 2:1 (other ratios can also be set). Therefore, the nine anchor boxes x are (8*8, 8*16, 16*8, 16*16, 16*32, 32*16, 32*32, 32*64, and 64*32).

[0070] Furthermore, an embodiment of the present invention classifies and identifies the content of the feature maps framed in all the obtained anchor frames through the detection classification layer and the detection frame regression layer of the region generation network, determines whether the feature map is the foreground or the background, and obtains the coordinate information of the anchor frame, and further selects the anchor frame whose feature map belongs to the foreground as the candidate frame, and displays the candidate frame on the feature map according to the corresponding coordinate value.

[0071] For further information, see Figure 4 As shown, in the embodiment of the present invention, the method of using the regional feature aggregation algorithm to pool the candidate boxes into a fixed size to obtain a standard candidate box includes:

[0072] S24, dividing each candidate box into n*n fixed-size units;

[0073] S25. Determine a sampling point in each unit according to a preset rule, calculate the pixel value of the sampling point using a bilinear interpolation method, and perform a maximum pooling operation on each block to select the pixel with the largest pixel value among the sampling points;

[0074] S26. Obtain a standard candidate box corresponding to each candidate box according to the selected pixel points.

[0075] An embodiment of the present invention performs a maximum pooling operation on each of the units to select the pixel with the largest pixel value among the sampling points, retains the candidate box containing the pixel with the largest pixel value, and eliminates the candidate box not containing the pixel with the largest pixel value to obtain the standard candidate box.

[0076] The bilinear interpolation method is to perform linear interpolation in two directions respectively, use the intersection point generated by the two linear interpolations as a sampling point and determine the pixel value of the sampling point.

[0077] S3. Regress and classify the standard candidate frame to obtain the target object candidate frame.

[0078] For details, see Figure 5 As shown, the standard candidate frame is regressed and classified to obtain the target object candidate frame, including:

[0079] S30, using a box regression function to obtain a predicted value of the offset of the standard candidate box relative to the actual position, so as to correct the standard candidate box;

[0080] S31. Input the standard candidate frame into the fully connected layer and softmax function in the pre-trained neural network, calculate the category to which the feature map in the standard candidate frame belongs, and obtain the target detection frame.

[0081] In the embodiment of the present invention, the standard candidate box is generally represented by a four-dimensional vector (x, y, w, h), which respectively represents the center point coordinates (x, y), width w, and height h of the standard candidate box. A is used to represent the standard candidate box. In the embodiment of the present invention, the box regression function is used to find the transformation relationship F, so that the standard candidate box is corrected to obtain the actual candidate box G, that is:

[0082] Given A=(A x , A y , A w , A h ), G=(G x , G y , G w , G h ), F(A x , A y , A w , A h )=(G x , G y , G w , G h );

[0083] F(A)=G is achieved by translation and scaling;

[0084] Pan: G x =A x +A w ·d x (A), G y =A y +A h ·d y (A);

[0085] Zoom: G w =A w ·exp(d w (A)), G h =A h ·exp(d h(A));

[0086] The embodiment of the present invention uses the box regression function to calculate the d x (A), d y (A), d w (A), d h (A) This achieves the correction of the standard candidate frame.

[0087] Furthermore, an embodiment of the present invention inputs the modified standard candidate frame into the fully connected layer and softmax function in a pre-trained neural network, calculates the category to which the feature map in the standard candidate frame belongs, and obtains a target detection frame. For example, one embodiment of the present invention classifies the feature map in the standard candidate frame into types such as cars, street lights, signs, normal driving lanes, and emergency lanes, from which a standard candidate frame classified as an emergency lane is obtained as the target detection frame.

[0088] S4. Perform coordinate mapping of the image to be detected according to the target candidate frame, and mark the target detection result in the image to be detected.

[0089] The embodiment of the present invention performs coordinate mapping to map the target object candidate frame to the image to be detected, so as to mark the identified target, such as the emergency lane, in the image to be detected.

[0090] The embodiments of the present invention use a pre-built feature extraction network to extract features from the image to be detected, enhancing the ability to express features in difficult scenarios, thereby improving the overall accuracy of object detection, such as emergency lane detection. At the same time, the region generation network and the region feature set algorithm are used to generate candidate frames, and these candidate frames are regressed and classified, effectively alleviating the problem of pixel deviation and improving the regression positioning of the candidate frames for object detection, thereby further improving the overall accuracy of object detection. Therefore, the embodiments of the present invention can improve the accuracy of object detection.

[0091] like Figure 6 FIG. 1 is a schematic diagram of a module of a target object detection device according to the present invention.

[0092] The object detection device 100 described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the object detection device 100 may include an image feature extraction module 101, a candidate box generation module 102, a classification regression module 103, and a candidate box mapping module 104. The modules described herein, also referred to as units, are a series of computer program segments that can be executed by an electronic device processor and perform a fixed function, and are stored in the electronic device's memory.

[0093] In this embodiment, the functions of each module / unit are as follows:

[0094] The image feature extraction module 101 is used to perform size standardization processing on the image to be detected to obtain a standard image, and extract image features from the standard image through a pre-built feature extraction network to obtain a feature image.

[0095] In an embodiment of the present invention, the image feature extraction module 101 can use various optical camera devices to obtain the image to be detected, such as using a traffic monitoring camera to photograph the emergency lane at specified time intervals, and uploading the photographed emergency lane images to a database, and finally summarizing the emergency lane images in the database to obtain the image to be detected.

[0096] Specifically, the image feature extraction module 101 performs size normalization processing on the image to be detected by the following operations:

[0097] Step A: Determine whether the size of the image to be detected is larger than the standard image size input by the user;

[0098] Step B: When the size of the image to be detected is larger than the standard image size, cropping the image to be detected according to the standard image size to obtain a standard image;

[0099] Step C: When the size of the image to be detected is smaller than the standard image size, padding the image to be detected according to the standard image size to obtain a standard image.

[0100] For example, the standard image size can be set to 1000*600. When the size of the image to be detected is 1200*1200, the image feature extraction module 101 can use a center cropping method to crop the image with a length of 1000 and a width of 600, taking the center of the image as the origin, to obtain a standard image; when the size of the image to be detected is 800*400, the image feature extraction module 101 can use a border filling method to expand outward with a preset pixel value using the image border as the boundary until the expanded size is 1000 in length and 600 in width, thereby obtaining a standard image.

[0101] In an embodiment of the present invention, the feature extraction network may be a DarkNet63 network. Furthermore, in one embodiment of the present invention, the extracting of image features from the standard image by a pre-constructed feature extraction network also includes constructing the feature extraction network before obtaining the feature image.

[0102] In detail, the feature extraction network is constructed by the following method: a first convolutional layer and a second convolutional layer, the first convolutional layer is constructed according to the convolution operation (Conv), the normalization operation (BN), and the activation operation (Leaky relu); the first convolutional layer is combined using the merging function (Concat) and the addition function (Add) to construct the second convolutional layer; the feature extraction network is constructed according to the first convolutional layer and the second convolutional layer.

[0103] Among them, the convolution operation is a 2D convolution operation, which is used to convolve the feature map from the standard image using 2D convolution kernels with different functions; the normalization operation is used to reduce the pixel values ​​of the pixels in the feature map using a normalization function; the activation operation uses an activation function to reduce the area size of the feature map; the merge function is used to connect two or more first convolution layers, and the add function is used to perform addition on the first convolution layer composition process.

[0104] The candidate box generation module 102 is used to perform target detection on the feature image using a region generation network, generate candidate boxes based on the detection results, and pool the candidate boxes into a fixed size using a region feature aggregation algorithm to obtain a standard candidate box.

[0105] In an embodiment of the present invention, the region proposal network (RPN) is used to select candidate boxes from the feature image.

[0106] Specifically, the candidate frame generation module 102 performs target detection on the feature image and generates a candidate frame based on the detection result through the following operations:

[0107] Step a: For each point on the feature image, generate a preset number of anchor boxes with different scales and aspect ratios;

[0108] Step b: input the anchor point frame into the detection and classification layer of the region generation network for classification, and determine whether the feature map in the anchor point frame belongs to the foreground or the background;

[0109] Step c: inputting the anchor box into the detection box regression layer of the region generation network to obtain the coordinate information of the anchor box;

[0110] Step d: Select the anchor box belonging to the foreground of the feature map as the candidate box, and display the candidate box on the feature map according to the corresponding coordinate value.

[0111] In an embodiment of the present invention, for each point on the feature image, the candidate box generation module 102 can generate 9 anchor boxes with different scales and aspect ratios. The 9 anchor boxes are obtained from 3 different sizes and 3 different ratios. For example, the 3 sizes are 8, 16, and 32 (other sizes can also be set), and the 3 different ratios are 1:1, 1:2, and 2:1 (other ratios can also be set). Therefore, the 9 anchor boxes x obtained are (8*8, 8*16, 16*8, 16*16, 16*32, 32*16, 32*32, 32*64, and 64*32).

[0112] Furthermore, the candidate frame generation module 102 described in the embodiment of the present invention classifies and identifies the feature map contents framed in all the obtained anchor frames through the detection classification layer and the detection frame regression layer of the region generation network, determines whether the feature map is the foreground or the background, and obtains the coordinate information of the anchor frame, and further selects the anchor frame whose feature map belongs to the foreground as the candidate frame, and displays the candidate frame on the feature map according to the corresponding coordinate value.

[0113] Furthermore, in an embodiment of the present invention, the method of using a regional feature aggregation algorithm to pool the candidate boxes into a fixed size to obtain a standard candidate box includes:

[0114] Step e: Divide each candidate box into n*n fixed-size units;

[0115] Step f, determining a sampling point in each unit according to a preset rule, calculating a pixel value of the sampling point using a bilinear interpolation method, and performing a maximum pooling operation on each block to select a pixel with the largest pixel value among the sampling points;

[0116] Step g: obtaining a standard candidate frame corresponding to each candidate frame based on the selected pixel points.

[0117] An embodiment of the present invention performs a maximum pooling operation on each of the units to select the pixel with the largest pixel value among the sampling points, retains the candidate box containing the pixel with the largest pixel value, and eliminates the candidate box not containing the pixel with the largest pixel value to obtain the standard candidate box.

[0118] The bilinear interpolation method is to perform linear interpolation in two directions respectively, use the intersection point generated by the two linear interpolations as a sampling point and determine the pixel value of the sampling point.

[0119] The classification and regression module 103 is used to regress and classify the standard candidate frames to obtain target object candidate frames.

[0120] In detail, the classification and regression module 103 regresses and classifies the standard candidate frame by the following method to obtain the target object candidate frame: using the frame regression function to obtain the offset prediction value of the standard candidate frame relative to the actual position to correct the standard candidate frame; inputting the standard candidate frame into the fully connected layer and softmax function in the pre-trained neural network, calculating the category to which the feature map in the standard candidate frame belongs, and obtaining the target detection frame.

[0121] In the embodiment of the present invention, the standard candidate box is generally represented by a four-dimensional vector (x, y, w, h), which respectively represents the center point coordinates (x, y), width w, and height h of the standard candidate box. The classification regression module 103 uses A to represent the standard candidate box. In the embodiment of the present invention, the box regression function is used to find the transformation relationship F, so that the standard candidate box is corrected to obtain the actual candidate box G, that is:

[0122] Given A=(A x , A y , A w , A h ), G=(G x , G y , G w , G h ), F(A x , A y , A w , A h )=(G x , G y , G w , G h );

[0123] F(A)=G is achieved by translation and scaling;

[0124] Pan: G x =A x +A w ·d x (A), G y =A y +A h ·d y (A);

[0125] Zoom: G w =A w ·exp(d w (A)), G h =A h ·exp(d h (A));

[0126] The classification regression module 103 of the embodiment of the present invention uses the box regression function to calculate the d x (A), dy (A), d w (A), d h (A) This achieves the correction of the standard candidate frame.

[0127] Furthermore, the classification and regression module 103 described in the embodiment of the present invention inputs the modified standard candidate frame into the fully connected layer and softmax function in the pre-trained neural network, calculates the category to which the feature map in the standard candidate frame belongs, and obtains the target detection frame. For example, one embodiment of the present invention classifies the feature map in the standard candidate frame into types such as cars, street lights, signs, normal driving lanes, and emergency lanes, and obtains the standard candidate frame classified as the emergency lane as the target detection frame.

[0128] The candidate frame mapping module 104 is configured to perform coordinate mapping on the image to be detected according to the target candidate frame, and mark the target detection result in the image to be detected.

[0129] The candidate box mapping module 104 of the embodiment of the present invention performs coordinate mapping to map the target object candidate box to the image to be detected, so as to mark the identified target, such as the emergency lane, in the image to be detected.

[0130] like Figure 7 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing the target object detection method according to the present invention.

[0131] The electronic device 1 may include a processor 10 , a memory 11 and a bus, and may further include a computer program stored in the memory 11 and executable on the processor 10 , such as an object detection program 12 .

[0132] Wherein, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Furthermore, the memory 11 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of a target detection program 12, etc., but can also be used to temporarily store data that has been output or is to be output.

[0133] In some embodiments, the processor 10 may be composed of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines. It executes or executes programs or modules stored in the memory 11 (such as executing a target detection program) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0134] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable connection and communication between the memory 11 and at least one processor 10, etc.

[0135] Figure 7 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 7The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0136] For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for powering the various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charging management, discharging management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device 1 may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0137] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0138] Optionally, the electronic device 1 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0139] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0140] The object detection program 12 stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When executed in the processor 10, it can achieve the following:

[0141] Obtain a picture to be detected, perform size normalization processing on the picture to be detected to obtain a standard picture, and extract picture features from the standard picture through a pre-built feature extraction network to obtain a feature picture;

[0142] Performing target detection on the feature image using a region generation network, generating candidate boxes based on the detection results, and pooling the candidate boxes into a fixed size using a region feature aggregation algorithm to obtain standard candidate boxes;

[0143] Regressing and classifying the standard candidate frame to obtain a target object candidate frame;

[0144] Coordinate mapping of the image to be detected is performed according to the target object candidate frame, and a target object detection result is marked in the image to be detected.

[0145] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figures 1 to 5 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0146] Furthermore, if the modules / units integrated into the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0147] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0148] Obtain a picture to be detected, perform size normalization processing on the picture to be detected to obtain a standard picture, and extract picture features from the standard picture through a pre-built feature extraction network to obtain a feature picture;

[0149] Performing target detection on the feature image using a region generation network, generating candidate boxes based on the detection results, and pooling the candidate boxes into a fixed size using a region feature aggregation algorithm to obtain standard candidate boxes;

[0150] Regressing and classifying the standard candidate frame to obtain a target object candidate frame;

[0151] Coordinate mapping of the image to be detected is performed according to the target object candidate frame, and a target object detection result is marked in the image to be detected.

[0152] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0153] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0154] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0155] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0156] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0157] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0158] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Second-order terms are used to indicate names and do not imply any particular order.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A target object detection method, characterized in that: The method comprises: Using a traffic monitoring camera to photograph the emergency lane at specified intervals to obtain a picture to be detected, performing size normalization processing on the picture to be detected to obtain a standard picture, and extracting picture features from the standard picture using a pre-built feature extraction network to obtain a feature picture; For each point on the feature image, generate a preset number of anchor boxes with different scales and aspect ratios, input the anchor boxes into the detection box classification layer of the region generation network for classification, determine whether the feature map in the anchor box belongs to the foreground or the background, select the anchor box whose feature map belongs to the foreground as the candidate box, divide each candidate box into n*n fixed-size units, determine sampling points in each unit according to preset rules, calculate pixel values ​​of the sampling points using bilinear interpolation, perform a maximum pooling operation on the pixel values ​​of the sampling points to select the pixel point with the largest pixel value among the sampling points, and obtain a standard candidate box corresponding to each candidate box based on the selected pixel points; Regress and classify the standard candidate frames to obtain the car type, street light type, sign type, normal driving lane type, and emergency lane type, and use the standard candidate frames corresponding to the emergency lane types as the target detection frames; Coordinate mapping of the image to be detected is performed according to the target object candidate frame, and a target object detection result is marked in the image to be detected.

2. The target object detection method according to claim 1, wherein: The performing size standardization processing on the image to be detected to obtain a standard image includes: Determine whether the size of the image to be detected is larger than the standard image size input by the user; When the size of the image to be detected is larger than the standard image size, cropping the image to be detected according to the standard image size to obtain a standard image; When the size of the image to be detected is smaller than the standard image size, padding processing is performed on the image to be detected according to the standard image size to obtain a standard image.

3. The target object detection method according to claim 1, wherein: After selecting the anchor box of the feature map belonging to the foreground as the candidate box, the method further includes: Inputting the anchor box into the detection box regression layer of the region generation network to obtain the coordinate information of the anchor box; The candidate frame is displayed on the feature image according to the coordinate values ​​corresponding to the coordinate information.

4. The target object detection method according to claim 1, wherein: The standard candidate frame is regressed and classified to obtain the car type, street light type, sign type, normal driving lane type, and emergency lane type, including: Using a box regression function to obtain a predicted value of the offset of the standard candidate box relative to the actual position, so as to correct the standard candidate box; The standard candidate frame is input into the fully connected layer and softmax function in the pre-trained neural network, and the category to which the feature map in the standard candidate frame belongs is calculated to obtain the car type, street light type, sign type, normal driving lane type, and emergency lane type.

5. The target object detection method according to any one of claims 1 to 4, wherein: Before extracting the image features from the standard image through the pre-built feature extraction network to obtain the feature image, the method further includes: Construct the first convolution layer according to the convolution operation, normalization operation, and activation operation; Use the merge function and the add function to construct the second convolution layer; The feature extraction network is constructed according to the first convolutional layer and the second convolutional layer.

6. The target object detection method according to any one of claims 1 to 5, wherein: The target object is the application lane.

7. An intelligent question-answering device based on big data, characterized in that: The device comprises: An image feature extraction module is configured to use a traffic monitoring camera to capture the emergency lane at specified intervals to obtain a to-be-detected image, perform size normalization on the to-be-detected image to obtain a standard image, and extract image features from the standard image using a pre-built feature extraction network to obtain a feature image; a candidate box generation module, configured to generate a preset number of anchor boxes with different scales and aspect ratios for each point on the feature image, input the anchor boxes into the detection box classification layer of the region generation network for classification, determine whether the feature map in the anchor box belongs to the foreground or the background, select the anchor box whose feature map belongs to the foreground as the candidate box, divide each candidate box into n*n fixed-size units, determine sampling points in each unit according to preset rules, calculate the pixel values ​​of the sampling points using bilinear interpolation, perform a maximum pooling operation on the pixel values ​​of the sampling points to select the pixel point with the largest pixel value among the sampling points, and obtain a standard candidate box corresponding to each candidate box based on the selected pixel points; A classification and regression module is used to regress and classify the standard candidate frames to obtain the car type, street light type, sign type, normal driving lane type, and emergency lane type, and use the standard candidate frames corresponding to the emergency lane types as target detection frames; The candidate box mapping module is used to perform coordinate mapping on the image to be detected according to the target candidate box, and mark the target detection result in the image to be detected.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the object detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium comprising a data storage area and a program storage area, characterized in that: The data storage area stores created data, and the program storage area stores a computer program; wherein, when the computer program is executed by a processor, the target object detection method according to any one of claims 1 to 6 is implemented.

Images