A target detection method and related apparatus

Abstract

The application discloses a target detection method and related device, the method comprises the following steps: acquiring a target image comprising a target object, performing feature extraction on the target image to obtain initial features corresponding to the target image; acquiring a plurality of reference sampling points matched with the initial features, determining sampling point offset information matched with each reference sampling point; based on the reference sampling points and the sampling point offset information, adjusting the dimensions of the initial features to obtain adjusted reference features; updating the reference features to the initial features, returning to the step of acquiring a plurality of reference sampling points matched with the initial features, until reference features matched with multiple dimensions are obtained, and based on all the reference features matched with multiple dimensions, acquiring a target detection result matched with the target image. In the above manner, the application can improve the accuracy of target detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a target detection method and related apparatus. Background Technology

[0002] In the field of computer vision, object detection is widely used. This technique locates and identifies objects in an image by predicting horizontal or rotated bounding boxes that can surround the target. However, this method of representing objects by bounding boxes has inherent geometric limitations: the actual outlines of many targets are not regular rectangles, especially for objects with complex shapes, such as objects with extreme aspect ratios. The bounding boxes often contain a large number of irrelevant background pixels or cannot closely fit the target edges, resulting in insufficient matching between the area inside the bounding box and the actual area of ​​the target.

[0003] Therefore, improving the accuracy of target detection has become an urgent problem to be solved. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a target detection method and related apparatus that can improve the accuracy of target detection.

[0005] To address the aforementioned technical problems, this application provides a target detection method comprising: acquiring a target image including a target object; extracting features from the target image to obtain initial features corresponding to the target image; acquiring multiple reference sampling points matching the initial features; determining sampling point offset information matching each reference sampling point; adjusting the dimensions of the initial features based on the reference sampling points and the sampling point offset information to obtain adjusted reference features; updating the reference features to the initial features; returning to the step of acquiring multiple reference sampling points matching the initial features, until reference features matching multiple dimensions are obtained; and acquiring a target detection result matching the target image based on the reference features matching all dimensions.

[0006] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a crane boom detection method, comprising: acquiring a target crane boom image including a crane boom to be detected; extracting features from the target crane boom image to obtain initial features corresponding to the target crane boom image; wherein, the aspect ratio of the crane boom to be detected in the target crane boom image is greater than a preset threshold; acquiring multiple reference sampling points matching the initial features, and determining sampling point offset information matching each reference sampling point; adjusting the dimensions of the initial features based on the reference sampling points and the sampling point offset information to obtain adjusted reference features; updating the reference features to the initial features, returning to the step of acquiring multiple reference sampling points matching the initial features, until reference features matching multiple dimensions are obtained; and acquiring a crane boom detection result matching the target crane boom image based on the reference features matching all dimensions.

[0007] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide an electronic device, including a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the method mentioned in the above technical solution.

[0008] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium having program instructions stored thereon, wherein the program instructions, when executed by a processor, implement the method mentioned in the above technical solution.

[0009] The beneficial effects of this application are as follows: Unlike existing technologies, the object detection method proposed in this application obtains the initial features corresponding to the target image, determines the reference sampling points that match the initial features in the current adjustment round, and the sampling point offset information matching each reference sampling point. Based on the reference sampling points and sampling point offset information, the method focuses on the region within the contour of the target object in the target image and obtains reference features after further feature extraction. After multiple adjustment rounds, reference features matching in different dimensions are obtained. Combining all reference features, a target detection result matching the target image is obtained. This target detection result accurately labels the region where the target object is located, greatly improving the target detection accuracy. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart illustrating one embodiment of the target detection method of this application; Figure 2 yes Figure 1 The flowchart of step S102 corresponds to another embodiment; Figure 3 This is a schematic diagram of one embodiment of the target detection model of this application; Figure 4 This is a structural schematic diagram of an embodiment corresponding to the first optimized submodule of this application; Figure 5 yes Figure 1 The flowchart of step S103 corresponds to another embodiment; Figure 6 yes Figure 5 The flowchart of step S302 corresponds to another embodiment; Figure 7 This is a schematic diagram of the structure of one embodiment corresponding to the second optimized submodule of this application; Figure 8 yes Figure 1 The flowchart of step S104 corresponds to another embodiment; Figure 9 This is a flowchart illustrating one implementation method of the object detection model training method; Figure 10 yes Figure 8 The flowchart of step S602 corresponds to another embodiment; Figure 11 This is a flowchart illustrating one embodiment of the boom detection method of this application; Figure 12 This is a schematic diagram of the structure of one embodiment of the electronic device of this application; Figure 13 This is a schematic diagram of one embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments, and different embodiments can be adaptively combined. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0012] Please see Figure 1 , Figure 1 This is a flowchart illustrating one embodiment of the target detection method of this application. The method includes: S101: Obtain the target image including the target object, extract features from the target image, and obtain the initial features corresponding to the target image.

[0013] In one embodiment, a target image for target detection is acquired, the target image including at least one target object. Preliminary feature extraction is performed on the acquired target image to obtain corresponding initial features.

[0014] In some implementation scenarios, in response to the presence of a target object in the target scene, the aforementioned target image can be obtained by capturing images of the target scene using a camera device.

[0015] In some implementation scenarios, after acquiring the target image, at least one round of convolution processing is performed on the target image to extract the initial features corresponding to the target image.

[0016] In one embodiment, in response to acquiring the video stream corresponding to the target scene, the target image may also be a video frame extracted from the video stream.

[0017] In some implementation scenarios, for video frames that match the target scene, keyframes are extracted from the video stream at preset intervals to serve as the target image. The target image is then subjected to convolution processing to extract the corresponding initial features.

[0018] S102: Obtain multiple reference sampling points that match the initial features, and determine the sampling point offset information that matches each reference sampling point.

[0019] In one embodiment, during the current round of deep feature extraction on the initial features, the initial features corresponding to the target image are traversed to obtain multiple reference sampling points that match the initial features. The initial features are then combined to predict the offset information of the sampling point that matches each reference sampling point.

[0020] S103: Based on the reference sampling points and sampling point offset information, the initial features are dimensionally adjusted to obtain the adjusted reference features.

[0021] In one embodiment, the distribution of all reference sampling points is adjusted based on the offset information of the reference sampling points and their matching sampling points, so that the adjusted reference sampling points are offset towards the region enclosed by the outline of the target object. Using the offset reference sampling points, deep feature extraction is performed on the initial features for the current round to obtain the dimension-adjusted reference features.

[0022] S104: Update the reference features to the initial features, return to the step of obtaining multiple reference sampling points that match the initial features, until reference features that match multiple dimensions are obtained, and obtain the target detection result that matches the target image based on the reference features that match all dimensions.

[0023] In one embodiment, after completing the deep feature extraction of the current round, the obtained reference features are updated to the initial features, and the process returns to the step of obtaining multiple sampling points that match the initial features, i.e., returning to step S102 above and executing subsequent steps sequentially. This continues until reference features matching multiple different dimensions are obtained, and all reference features are decoded to obtain the target detection result matching the target image.

[0024] In a specific application scenario, after extracting initial features from the target image, three rounds of deep feature extraction are performed based on these initial features to obtain reference features for each round. Decoding and detection are then performed on each reference feature to obtain a target detection result that matches the target image. The input to the first round of deep feature extraction is the initial features obtained from feature extraction of the target image, while the input to the subsequent rounds of deep feature extraction is the output of the previous round.

[0025] The object detection method proposed in this application obtains the initial features corresponding to the target image, determines the reference sampling points that match the initial features in the current adjustment round, and the sampling point offset information of each reference sampling point. Based on the reference sampling points and sampling point offset information, the method focuses on the region within the contour of the target object in the target image and obtains reference features after further feature extraction. After multiple adjustment rounds, reference features matching in different dimensions are obtained. Combining all reference features, an object detection result matching the target image is obtained. This object detection result accurately labels the region where the target object is located, greatly improving the object detection accuracy.

[0026] Please see Figure 2 and Figure 3 , Figure 2 yes Figure 1 The flowchart of step S102 corresponds to another embodiment. Figure 3 This is a schematic diagram of one embodiment of the object detection model of this application. The above object detection method is implemented using a trained object detection model, which includes a feature optimization module matched to each dimension. The implementation process of step S102 includes: S201: Input the initial features into the first optimization submodule in the feature optimization module, and use the first optimization submodule to obtain multiple reference sampling points that match the initial features.

[0027] In one embodiment, the initial features are input into the first optimization submodule of the feature optimization module in the current adjustment round, so as to use the first optimization submodule to traverse each pixel in the initial features to obtain multiple reference sampling points.

[0028] In some implementation scenarios, for the feature optimization module corresponding to the current adjustment round, the convolution kernel matched by the first optimization sub-module is determined. The convolution kernel is then used to traverse the initial features to determine multiple reference sampling points. These multiple reference sampling points include the center point of the convolution kernel and other points surrounding the center point of the convolution kernel.

[0029] In a specific application scenario, when the convolution kernel size is 3×3, and the coordinates of the kernel center point are... At that time, the set consisting of multiple reference sampling points is .

[0030] S202: Based on the initial features and their matching reference sampling points, obtain the sampling point offset information output by the first optimization submodule that matches each reference sampling point; wherein, the sampling point offset information includes the position offset and the offset weight.

[0031] In one embodiment, based on the acquired multiple reference sampling points, the first optimization submodule predicts the sampling point offset information matching the reference sampling points according to the initial features, so that in the subsequent process of deep extraction of the initial features, more attention is paid to the area where the target object itself is located.

[0032] In some implementation scenarios, the above sampling point offset information includes the position offset and the offset weight.

[0033] For a specific application scenario, please refer to Figure 4 , Figure 4 This is a structural diagram of one embodiment corresponding to the first optimized submodule of this application. For example... Figure 4 As shown, the first optimization submodule includes multiple deformable convolutional layers, namely a first deformable convolutional layer, a second deformable convolutional layer, a third deformable convolutional layer, and a fourth deformable convolutional layer. The acquired initial features are input into the first and second deformable convolutional layers in this first optimization submodule to obtain the first offset detection result output by the first deformable convolutional layer and the second offset detection result output by the second deformable convolutional layer. The third deformable convolutional layer performs deep extraction on the first offset detection result to obtain the output third offset detection result. The second and third offset detection results are then fused to obtain the fused offset detection result. The fused offset detection result is input into the fourth deformable convolutional layer to obtain highly accurate sampling point offset information matching each reference sampling point. Each offset detection result includes a reference sampling point matching the initial features and sampling point offset information. By utilizing multiple deformable convolutional layers for multi-round detection of sampling point offsets, the accuracy of determining the sampling point offset information is greatly improved.

[0034] Additionally, it should be noted that, Figure 3The diagram only schematically illustrates the structure of the first feature optimization module, which sequentially includes a first optimization submodule and a second optimization submodule; the structures of the remaining feature optimization modules are identical. Secondly, the object detection model also includes a feature enhancement module. This module enhances the reference features output by the last feature optimization module to improve feature representation. This feature enhancement module sequentially includes a pooling layer and an attention layer. Additionally, the object detection model includes multiple upsampling modules and residual modules. These upsampling and residual modules are used to enhance the reference features output by different feature optimization modules. Their specific implementation can be referenced from existing neural network models, such as the YOLO (You Only Look Once) model.

[0035] The above scheme utilizes the feature optimization module corresponding to the current adjustment round to determine multiple reference sampling points and sampling point offset information that match the initial features under the current adjustment round. This enables subsequent deep feature extraction of the current adjustment round to be performed using the reference sampling points and sampling point offset information, resulting in reference features that have a high degree of fit with the contour region of the target object.

[0036] Please see Figure 5 , Figure 5 yes Figure 1 The flowchart of step S103 corresponds to another embodiment. Specifically, the implementation process of step S103 includes: S301: Based on the reference sampling point and its matching sampling point offset information, determine the local receptive field in the initial feature that matches the contour of the target object.

[0037] In one implementation, feature regions corresponding to the target object contour are obtained from the initial features based on the position offset and offset weight corresponding to each reference sampling point. Based on the feature regions, local receptive fields matching the initial features are determined.

[0038] In some implementation scenarios, the distribution of all reference sampling points is adjusted based on the positional offset and offset weight in the sampling point offset information, so that the adjusted reference sampling points are offset towards the target object contour area. Based on the adjusted distribution of all reference sampling points, the feature region related to the target object contour is determined, and this feature region is used as the local receptive field for matching the initial feature in the current adjustment round.

[0039] S302: Use the local receptive field to extract features from the initial features to obtain reference features after dimension adjustment.

[0040] In one embodiment, feature extraction is performed on the initial features using a defined local receptive field to obtain dimension-adjusted reference features.

[0041] In some implementation scenarios, feature extraction is performed on the initial features using a defined local receptive field. The specific formula for feature extraction based on the current convolutional kernel position is as follows:

[0042] in, This represents the center reference sampling point corresponding to the current convolution kernel. Indicates initial features, This indicates the sampling point offset information. Indicates the offset weight. This indicates an adjustable parameter. This represents the result of feature extraction from the initial features using the current convolution kernel.

[0043] The above scheme utilizes sampling point offset information to shift multiple reference sampling points towards the region enclosed by the target object's contour, thereby determining the corresponding local receptive field. Deep feature extraction is then performed on the initial features using this local receptive field to obtain dimension-adjusted reference features.

[0044] Please see Figure 3 and Figure 6 , Figure 6 yes Figure 5 The flowchart of step S302 corresponds to another embodiment. The feature optimization module also includes a second optimization submodule coupled to the first optimization submodule. Based on this, the implementation process of step S302 includes: S401: The first optimization submodule is used to extract features from the initial features based on the local receptive field to obtain the first optimized features corresponding to the initial features.

[0045] In one embodiment, after determining the local receptive field, the first optimization submodule is used to extract features from the initial features based on the local receptive field to obtain the first optimized features corresponding to the initial features.

[0046] In some implementation scenarios, local receptive fields are used to extract local features from the initial features in order to obtain the first optimized features that are more relevant to the region where the target object is located.

[0047] In a specific application scenario, a pre-defined convolutional kernel is used to extract features from the initial features while keeping the input and output dimensions unchanged, so as to obtain the first optimized features after feature enhancement.

[0048] S402: Input the first optimized feature into the second optimized submodule, and use multiple convolutional networks in the second optimized submodule to process the first optimized feature respectively to obtain multiple second optimized features of different dimensions; wherein, the convolutional kernel size corresponding to different convolutional networks is different.

[0049] In one embodiment, in response to the coupling of a first optimization submodule and a second optimization submodule within the same feature optimization module, the first optimization submodule inputs a first optimized feature to the second optimization submodule. In response to the second optimization submodule including multiple convolutional networks with different kernel sizes for each network, each convolutional network processes the first optimized feature separately to obtain multiple second optimized features in different dimensions.

[0050] For a specific application scenario, please refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of the second optimization submodule corresponding to one embodiment of this application. The second optimization submodule includes a first convolutional network, a second convolutional network, and a third convolutional network. The first convolutional network has a kernel size of 1×1 and the number of output channels is half the number of input channels; the second convolutional network has a kernel size of 3×3 and the number of output channels is the same as the number of input channels; the third convolutional network has a kernel size of 5×5 and the number of output channels is half the number of input channels. Responding to a feature map dimension of H×W×C for the first optimized feature input to the second optimization submodule, the first convolutional network processes the first optimized feature, and the output feature map dimension of the second optimized feature is H / 2×W / 2×C / 2; the feature map dimension of the second optimized feature output by the second convolutional network is H / 2×W / 2×C / 2; and the feature map dimension of the second optimized feature output by the third convolutional network is H / 2×W / 2×C / 2.

[0051] S403: Based on all the second optimized features, the reference features are obtained.

[0052] In one embodiment, after obtaining the second optimized features output by each convolutional network, all the second optimized features are fused to obtain fused features. A dimensionality transformation network is then used to transform the fused features to obtain reference features.

[0053] In a specific application scenario, such as Figure 7 As shown, the dimension transformation network includes 1×1 convolutional kernels, and the number of output channels of the dimension transformation network is twice the number of input channels of the second optimization submodule. The fused features are input into the dimension transformation network, which expands the feature channels of the fused features to obtain reference features. By utilizing the dimension transformation network to increase the feature dimension of the reference features, the feature representation capability of the reference features is enhanced.

[0054] In one embodiment, after obtaining the second optimized features output by each convolutional network in step S402, all the second optimized features are fused to obtain reference features.

[0055] Please see Figure 8 , Figure 8 yes Figure 1 The flowchart of step S104 corresponds to another implementation method. The target detection model also includes multiple decoding modules; therefore, the implementation process of step S104 includes: S501: Input each reference feature into the corresponding decoding module.

[0056] In one embodiment, the reference features obtained in different dimensions are input to the corresponding decoding module.

[0057] S502: In response to each decoding module having a corresponding reference size range, the decoding module decodes the input reference features and outputs the reference detection result located within the reference size range.

[0058] In one embodiment, for each decoding module, the input reference features are decoded to obtain candidate detection results that match the target category output by the decoding module. These candidate detection results are used to characterize the target region where the target object is located in the target image.

[0059] Furthermore, each decoding module is matched with a corresponding reference size range. For each decoding module, it is determined whether the target size of the target region corresponding to the candidate detection result is within the corresponding reference size range. If the target size of the target region corresponding to the candidate detection result is within the corresponding reference size range, the candidate size result is used as the reference detection result. All candidate detection results are traversed until a reference detection result that is within the corresponding reference size range is selected from all candidate detection results.

[0060] In a specific application scenario, the target detection model includes a first decoding module, a second decoding module, and a third decoding module. The first decoding module has a smaller anchor box size, suitable for detecting small-sized targets; the second decoding module has a larger anchor box size than the first decoding module, suitable for detecting medium-sized targets; and the third decoding module has the largest anchor box size, suitable for detecting large-sized targets. For all candidate detection results output by the first decoding module, the candidate detection results whose target region size falls within the reference size range of the first decoding module are used as reference detection results.

[0061] S503: Based on the reference detection results output by each decoding module, obtain the target detection results that match the target image.

[0062] In one embodiment, the reference detection results output by each decoding module are marked in the target image to obtain target detection results that match the target image, so as to intuitively display all target objects in the target image and the regions where each target object is located.

[0063] The above scheme, by setting multiple decoding modules in the target detection model to perform scale-based detection of the target image and cover the full-size targets in the target image, helps to improve the accuracy of detecting different target objects of different sizes in the target image.

[0064] Please see Figure 9 , Figure 9 This is a flowchart illustrating one implementation of the object detection model training method. Specifically, the training process of the object detection model mentioned in any of the above implementations includes: S601: Obtain multiple initial training images corresponding to the training object.

[0065] In one embodiment, multiple initial training images corresponding to the training objects are obtained for the actual application scenario of object detection.

[0066] In a specific application scenario, in order to perform target detection on the crane boom, the crane boom is used as the training object, and multiple initial training images including the crane boom are acquired.

[0067] S602: Enhancement processing is performed on at least a portion of the initial training images to obtain a target training image; wherein the target training image is matched with training labels, and the training labels are used to characterize the contour regions corresponding to the training objects in the target training image.

[0068] In one embodiment, the acquired initial training image is augmented to obtain a corresponding target training image. The target training image is then labeled with training objects to obtain a training label matching each target training image; this training label is used to characterize the contour region corresponding to the training object.

[0069] In some implementation scenarios, the enhancement processing methods include at least one of color adjustment and pose adjustment. Specifically, at least one of the hue, saturation, and brightness of the initial training image is adjusted to obtain the enhanced target training image. Alternatively, the initial training image is horizontally flipped to obtain the enhanced target training image. Or, the training object in the initial training image is angularly rotated to change the pose of the training object, resulting in the enhanced target training image.

[0070] In some implementation scenarios, training objects in the target training image are labeled using bounding boxes, using relevant labeled objects, to obtain training bounding boxes that fit the contours of the target objects in the target training image. These training bounding boxes are then used as the training labels for the corresponding target training images. It should be noted that, to improve the learning quality of the object detection model during training, the training bounding boxes need to have a high degree of fit with the contours of the training objects when labeling the target training images; that is, the training bounding boxes are not regular rectangles.

[0071] S603: Input the target training image into the target detection model and obtain the training detection results output by the target detection model.

[0072] In one embodiment, a target training image is input into a target detection model, enabling the model to detect the target training image and output the corresponding training detection result. The specific process of outputting the training detection result can be referred to the corresponding embodiment described above, and will not be elaborated upon here.

[0073] S604: Based on the training detection results and training labels, obtain the training loss that matches the target detection model, and use the training loss to adjust the parameters in the target detection model until the model convergence condition is met, thus obtaining the trained target detection model.

[0074] In one embodiment, for the target training image, based on the training detection results output by the target detection model and the corresponding training labels, the training loss of the target detection model in the current training round is calculated, and the parameters in the target detection model are adjusted using the training loss. Training stops when the training loss of the target detection model converges, or when the number of training rounds reaches a preset threshold, and the trained target detection model is obtained.

[0075] The above scheme constructs multiple target training images and their matching training labels. The training labels include training bounding boxes that closely match the contours of the training objects. The object detection model learns the association information between the training bounding boxes and the contours of the training objects in the target training images. This ensures that the trained object detection model outputs target bounding boxes with a high degree of fit to the target object contours during actual object detection, thus avoiding redundant areas such as background from entering the target bounding boxes and improving the accuracy of object detection.

[0076] Please see Figure 10 , Figure 10 yes Figure 9 The flowchart of step S602 corresponds to another embodiment. Specifically, the implementation process of step S602 includes: S701: Obtain preset transformation information that matches the initial training image; wherein the preset transformation information includes at least one of axial transformation rate and pixel retention rate.

[0077] In one embodiment, preset transformation information is determined, which includes at least one of axial transformation rate and pixel retention rate.

[0078] In some implementation scenarios, the axial transformation rate is used to axially stretch or shorten the initial training image. The pixel retention rate is used to randomly retain a portion of pixels in the initial training image to reduce its resolution, thereby improving the efficiency of subsequent training.

[0079] S702: Transform at least a portion of the initial training images using preset transformation information to obtain transformed training images.

[0080] In one embodiment, at least a portion of the initial training images are transformed using the aforementioned predetermined transformation information to obtain transformed training images.

[0081] In some implementation scenarios, in response to preset transformation information including axial transformation rate and pixel retention rate, where the axial transformation rate is 1.5 horizontally and the pixel retention rate is 0.9, the initial training image is stretched horizontally so that its horizontal length is 1.5 times that before stretching; and 10% of the pixels in the initial training image are randomly removed. This transformation process, by stretching the initial training image horizontally by 1.5 times to simulate training objects under extreme aspect ratios, helps improve the object detection model's learning of training objects under extreme aspect ratios.

[0082] S703: Obtain the target training image based on at least one transformed training image.

[0083] In one embodiment, to further increase the complexity of the target training image and improve the performance of the target detection model after training, at least one transformed training image obtained in step S703 is fused to obtain the target training image.

[0084] In some implementation scenarios, multiple transformed training images are stitched together to obtain a target training image that includes multiple training objects. Alternatively, multiple transformed training images are randomly scaled, and then the scaled transformed training images are stitched together to obtain the target training image.

[0085] S704: Obtain the corresponding training labels based on the contour regions of all training objects in the target training image.

[0086] In one embodiment, training objects in the target training image are labeled using relevant annotation objects in the form of annotation boxes to obtain training annotation boxes that fit the contours of the target objects in the target training image. These training annotation boxes are then used as the training labels for the corresponding target training images.

[0087] The above scheme, by pre-determining preset transformation information and using the preset transformation information to obtain target training images, simulates target detection tasks in complex scenarios and obtains corresponding target training images, which helps to improve the training quality of subsequent target detection models.

[0088] Please see Figure 11 , Figure 11 This is a flowchart illustrating one embodiment of the boom detection method of this application. Specifically, the boom detection method includes: S801: Acquire a target boom image including the boom to be detected, extract features from the target boom image to obtain the initial features corresponding to the target boom image; wherein, the aspect ratio of the boom to be detected in the target boom image is greater than a preset threshold.

[0089] In one embodiment, when the target object mentioned in any of the above embodiments is the crane boom to be detected, an image of the target crane boom including the crane boom to be detected is acquired, and feature extraction is performed on the target crane boom image to obtain initial features corresponding to the target crane boom image.

[0090] In some implementation scenarios, the target boom in the target boom image is an opposing boom with an extreme aspect ratio, where the aspect ratio is greater than a preset threshold. The process of acquiring the initial features corresponding to the target boom image can refer to the process of acquiring the initial features corresponding to the target image in the corresponding implementation described above; and the preset threshold can be set according to the actual scenario.

[0091] In a specific application scenario, based on the object characteristics of the boom to be detected, the above-mentioned preset threshold is set to 10, that is, in response to the ratio of the boom length to the boom width of the boom to be detected in the target boom image being greater than 10, the boom detection method proposed in this application is adopted to improve the accuracy of detecting the boom to be detected.

[0092] S802: Obtain multiple reference sampling points that match the initial features, and determine the sampling point offset information that matches each reference sampling point.

[0093] In one embodiment, after obtaining the initial features corresponding to the target boom image, the initial features are traversed to obtain multiple reference sampling points that match the initial features. Based on the initial features, the sampling point offset information matching each reference sampling point is predicted. The specific process for predicting the sampling point offset information matching each reference sampling point corresponding to the target boom image can be referred to the corresponding embodiment described above.

[0094] S803: Based on the reference sampling points and sampling point offset information, the initial features are dimensionally adjusted to obtain the adjusted reference features.

[0095] In one embodiment, after determining multiple reference sampling points and their matching sampling point offset information through each round of deep extraction, the contour region of the crane boom to be detected is focused based on the reference sampling points and their matching sampling point offset information to obtain the corresponding reference features. The process of obtaining the reference features can refer to the corresponding embodiment described above.

[0096] S804: Update the reference features to the initial features, return to the step of obtaining multiple reference sampling points that match the initial features, until reference features that match multiple dimensions are obtained, and obtain the crane detection result that matches the target crane image based on the reference features that match all dimensions.

[0097] In one embodiment, after completing the deep feature extraction for the current round, the reference features corresponding to the obtained target boom image are updated to the initial features, and the process returns to the step of obtaining multiple sampling points matching the initial features. That is, it returns to step S802 above and executes subsequent steps sequentially until reference features matching multiple different dimensions are obtained. All obtained reference features are integrated to obtain the target detection result, and the boom outline of each crane in the target boom image is labeled in the form of target bounding boxes. The specific implementation process can be referred to the corresponding embodiment above, and will not be described in detail here.

[0098] The above scheme, because the crane boom to be inspected has extremely slender and high aspect ratio geometric features, detects the target boom image including the crane boom to be inspected by the above boom detection method, so that the final target annotation box can closely fit the actual contour of the boom to be inspected, avoiding redundant background pixels such as sky in the target annotation box, thus improving the accuracy of detecting the crane boom to be inspected.

[0099] Please see Figure 12 , Figure 12This is a schematic diagram of one embodiment of the electronic device of this application. The electronic device includes a memory 10 and a processor 20 coupled to each other. The memory 10 stores program instructions, and the processor 20 executes the program instructions to implement the methods mentioned in any of the above embodiments. Specifically, the electronic device includes, but is not limited to, desktop computers, laptops, tablets, servers, etc., and is not limited thereto. In addition, the processor 20 may also be called a CPU (Center Processing Unit). The processor 20 may be an integrated circuit chip with signal processing capabilities. The processor 20 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor may be a microprocessor or any conventional processor. In addition, the processor 20 may be implemented by integrated circuit chips.

[0100] Please see Figure 13 , Figure 13 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 30 stores program instructions 40 that can be executed by a processor. When the program instructions 40 are executed by the processor, they implement the methods mentioned in any of the above embodiments.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A target detection method, characterized in that, include: Obtain a target image including the target object, perform feature extraction on the target image, and obtain the initial features corresponding to the target image; Obtain multiple reference sampling points that match the initial feature, and determine the sampling point offset information that matches each of the reference sampling points; Based on the reference sampling points and the sampling point offset information, the initial features are dimensionally adjusted to obtain the adjusted reference features; The reference features are updated to the initial features, and the process returns to the step of obtaining multiple reference sampling points that match the initial features, until reference features that match multiple dimensions are obtained. Based on the reference features that match all dimensions, the target detection result that matches the target image is obtained.

2. The target detection method according to claim 1, characterized in that, The object detection method is implemented using a trained object detection model, which includes a feature optimization module matching each dimension. The step of acquiring multiple reference sampling points matching the initial features and determining the sampling point offset information matching each reference sampling point includes: The initial features are input into the first optimization submodule in the feature optimization module, and the first optimization submodule is used to obtain a plurality of reference sampling points that match the initial features; Based on the initial features and their matching reference sampling points, the sampling point offset information output by the first optimization submodule that matches each of the reference sampling points is obtained; wherein, the sampling point offset information includes position offset and offset weight.

3. The target detection method according to claim 2, characterized in that, The step of adjusting the dimensions of the initial features based on the reference sampling points and the sampling point offset information to obtain the adjusted reference features includes: Based on the reference sampling points and their matching sampling point offset information, the local receptive field that matches the contour of the target object in the initial features is determined; The initial features are extracted using the local receptive field to obtain the dimension-adjusted reference features.

4. The target detection method according to claim 3, characterized in that, The step of determining the local receptive field in the initial features that matches the contour of the target object based on the reference sampling points and their matching sampling point offset information includes: Based on the position offset and offset weight corresponding to each of the reference sampling points, the feature region corresponding to the contour of the target object is obtained from the initial features; Based on the feature region, the local receptive field that matches the initial feature is determined.

5. The target detection method according to claim 3, characterized in that, The feature optimization module further includes a second optimization submodule coupled to the first optimization submodule, wherein the initial features are extracted using the local receptive field to obtain the dimension-adjusted reference features, including... The first optimization submodule is used to extract features from the initial features based on the local receptive field to obtain the first optimized features corresponding to the initial features; The first optimized feature is input into the second optimization submodule, and multiple convolutional networks in the second optimization submodule are used to process the first optimized feature to obtain multiple second optimized features of different dimensions; wherein, the convolutional kernel size is different for different convolutional networks; The reference features are obtained based on all the second optimized features.

6. The target detection method according to claim 2, characterized in that, The target detection model further includes multiple decoding modules. The step of obtaining target detection results matching the target image based on reference features matched across all dimensions includes: Each of the reference features is input to the corresponding decoding module; In response to each of the decoding modules being matched with a corresponding reference size range, the input reference features are decoded using the decoding modules, and a reference detection result located within the reference size range is output. Based on the reference detection results output by each of the decoding modules, the target detection result matching the target image is obtained.

7. The target detection method according to claim 2, characterized in that, The training process of the target detection model includes: Obtain multiple initial training images corresponding to the training object; Enhancement processing is performed on at least a portion of the initial training images to obtain a target training image; wherein the target training image is matched with training labels, and the training labels are used to characterize the contour regions corresponding to the training objects in the target training image; The target training image is input into the target detection model to obtain the training detection result output by the target detection model; Based on the training detection results and the training labels, a training loss matching the target detection model is obtained. The parameters in the target detection model are adjusted using the training loss until the model convergence condition is met, thus obtaining the trained target detection model.

8. The target detection method according to claim 7, characterized in that, The enhancement processing based on at least a portion of the initial training images to obtain the target training image includes: Obtain preset transformation information that matches the initial training image; wherein, the preset transformation information includes at least one of axial transformation rate and pixel retention rate; The preset transformation information is used to transform at least a portion of the initial training images to obtain the transformed training images. The target training image is obtained based on at least one of the transformed training images; Based on the contour regions corresponding to all the training objects in the target training image, the corresponding training labels are obtained.

9. A method for detecting a crane boom, characterized in that, include: A target boom image including the boom to be detected is acquired, and feature extraction is performed on the target boom image to obtain the initial features corresponding to the target boom image; wherein, the aspect ratio of the boom to be detected in the target boom image is greater than a preset threshold. Obtain multiple reference sampling points that match the initial feature, and determine the sampling point offset information that matches each of the reference sampling points; Based on the reference sampling points and the sampling point offset information, the initial features are dimensionally adjusted to obtain the adjusted reference features; The reference features are updated to the initial features, and the process returns to the step of obtaining multiple reference sampling points that match the initial features, until reference features that match multiple dimensions are obtained. Based on the reference features that match all dimensions, the crane detection result that matches the target crane image is obtained.

10. An electronic device, characterized in that, include: A memory and a processor are coupled to each other, the memory storing program instructions, and the processor executing the program instructions to implement the method as described in any one of claims 1-9.

11. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the method as described in any one of claims 1-9.

Images