Target detection method and device, equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a target detection method and device, equipment and a storage medium. The target detection method comprises the following steps: acquiring a detection image based on shooting equipment of a robot; inputting the detection image into a pre-trained deep learning model for detection to obtain a detection result; a network structure of the deep learning model comprises an input end, a Backbone module, a Neck module and a Detect head module which are connected in sequence, the Backbone module comprises a first Mmba block layer, a second Mmba block layer, a third Mmba block layer and a fourth Mmba block layer which are arranged in a stacked mode and connected in sequence, and all the Mmba block layers are connected with the Neck module; by means of the mode, the defects that an existing robot is low in working efficiency, low in robustness, low in precision, low in efficiency and large in computing power resource occupation when carrying out target detection in the dynamic environment can be overcome.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a target detection method, device, equipment and storage medium. Background Art

[0002] With the development of intelligent manufacturing technology, robots are widely used in industrial production, replacing manual workpiece handling and other tasks. In the prior art, a solution for detecting targets through computer vision processing has emerged. Through the target detection algorithm of computer vision, the position, shape and size information of the target can be accurately located.

[0003] At present, target detection algorithms are mostly deployed on fixed-position robotic arms. They need to collect photos of workpieces from different angles from multiple camera positions and feed them into the model for detection and comprehensive judgment. The process is cumbersome and the data throughput is large. In addition, target detection algorithms often use large models such as transformers to meet performance requirements, but they occupy a large amount of computing resources. Therefore, the target detection algorithms currently used in dynamic environments have low robustness, low accuracy, low efficiency, and high computing resource usage. They cannot meet the requirements of robots for target detection in dynamic environments, which need to overcome large occlusions, lighting changes, target deformation, scale changes, and detection angle changes. Summary of the invention

[0004] Based on this, the present invention provides a target detection method, device, equipment and storage medium, which can solve the shortcomings of existing robots in performing target detection in dynamic environments, such as low work efficiency, low robustness, low accuracy, low efficiency and large computing power resource occupation.

[0005] In order to solve the above technical problems, a technical solution adopted by the present invention is: to provide a target detection method, comprising:

[0006] Acquire detection images based on the robot’s shooting equipment;

[0007] The detection image is input into a pre-trained deep learning model for detection to obtain a detection result; the network structure of the deep learning model includes an input end, a Backbone module, a Neck module and a Detecthead module connected in sequence, wherein the Backbone module includes a first Mamba block layer, a second Mamba block layer, a third Mamba block layer and a fourth Mamba block layer which are stacked and connected in sequence, and each Mamba block layer is connected to the Neck module respectively;

[0008] The input end is used to input the detection image into a pre-trained deep learning model;

[0009] Each Mamba block layer in the Backbone module is used to downsample the detection image, and perform multi-directional scanning processing, channel transformation processing and size transformation processing on each sampling result, and output feature maps with different numbers of channels and different sizes;

[0010] The Neck module is used to extract multi-scale features from the feature map;

[0011] The Detect head module is used to make predictions based on the multi-scale features to obtain the detection results.

[0012] According to an embodiment of the present invention, the Backbone module further includes an image block embedding layer, a layer normalization layer, and an activation layer; before the first Mamba block layer performs the step of downsampling the detection image, performing multi-directional scanning processing, channel transformation processing, and size transformation processing on each sampling result, and outputting feature maps with different numbers of channels and different sizes, the following steps are also included:

[0013] Using the image block embedding layer to perform block processing and image conversion processing on the detection image and output it;

[0014] Using the layer normalization layer to perform data normalization processing on the output of the image block embedding layer and output it;

[0015] The activation layer is used to perform nonlinear transformation processing on the output of the normalization layer and output the resultant output.

[0016] According to one embodiment of the present invention, each Mamba block layer also includes: multiple scanning units, an information extraction unit corresponding to the scanning unit, an information fusion unit connected to the information extraction unit, a layer normalization unit connected to the information fusion unit, a multilayer perceptron connected to the layer normalization unit, and a residual processing unit, wherein the scanning directions of the multiple scanning units are different.

[0017] According to an embodiment of the present invention, each Mamba block layer performs the steps of performing multi-directional scanning processing, channel transformation processing and size transformation processing on each sampling result, and outputting feature maps with different numbers of channels and different sizes, and further includes the following steps:

[0018] Perform multi-directional scanning processing, channel transformation processing, and size transformation processing on the sampling results through a plurality of the scanning units;

[0019] The information extraction unit rearranges the output of the corresponding scanning unit and extracts the global feature information;

[0020] fusing the global feature information output by each of the information extraction units through the information fusion unit;

[0021] Normalizing the output of the information fusion unit by the layer normalization unit;

[0022] Introducing nonlinear changes to the output of the layer normalization unit through the multilayer perceptron;

[0023] The residual processing unit performs residual processing on the output of the information fusion unit and the output of the multi-layer perceptron.

[0024] According to an embodiment of the present invention, the rearranging the outputs of the corresponding scanning units and extracting the global feature information by the information extraction unit comprises the following steps:

[0025] Perform convolution and activation operations on the output of the corresponding scanning unit to obtain the latent state space features;

[0026] The global context information of the latent state space feature is selectively processed based on a preset selective structured state space model to obtain the global feature information.

[0027] According to one embodiment of the present invention, after the robot-based shooting device acquires the detection image and before the detection image is input into the pre-trained deep learning model for detection, the following steps are also included:

[0028] The detection image is preprocessed; the preprocessing is image enhancement processing.

[0029] According to one embodiment of the present invention, after obtaining the detection result, the following steps are also included:

[0030] The detection result is post-processed, and the post-processing is to determine whether to output the detection result based on the preset threshold value of each category.

[0031] In order to solve the above technical problems, another technical solution adopted by the present invention is: to provide a target detection device, which is applied to the target detection method, and the target detection device comprises:

[0032] An acquisition module, used for acquiring detection images based on a shooting device of the robot;

[0033] A detection module is used to input the detection image into a pre-trained deep learning model for detection to obtain a detection result; the network structure of the deep learning model includes an input end, a Backbone module, a Neck module and a Detect head module connected in sequence, wherein the Backbone module includes a first Mambablock layer, a second Mamba block layer, a third Mamba block layer and a fourth Mamba block layer which are stacked and connected in sequence, and each Mambablock layer is connected to the Neck module respectively;

[0034] The input end is used to input the detection image into a pre-trained deep learning model;

[0035] Each Mamba block layer in the Backbone module is used to downsample the detection image, and perform multi-directional scanning processing, channel transformation processing and size transformation processing on each sampling result, and output feature maps with different numbers of channels and different sizes;

[0036] The Neck module is used to extract multi-scale features from the feature map;

[0037] The Detect head module is used to make predictions based on the multi-scale features to obtain the detection results.

[0038] To solve the above technical problems, another technical solution adopted by the present invention is: to provide a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the target detection method is implemented when the processor executes the computer program.

[0039] In order to solve the above technical problem, another technical solution adopted by the present invention is: providing a computer storage medium, on which a computer program is stored, and the computer program implements the above target detection method when executed by a processor.

[0040] The beneficial effects of the present invention are as follows: a target detection method comprises: acquiring a detection image based on a shooting device of a robot; inputting the detection image into a pre-trained deep learning model for detection to obtain a detection result; the network structure of the deep learning model comprises an input end, a Backbone module, a Neck module and a Detect head module connected in sequence, wherein the Backbone module comprises a first Mamba block layer, a second Mambablock layer, a third Mamba block layer and a fourth Mamba block layer which are stacked and connected in sequence, and each Mamba block layer is respectively connected to the Neck module; down-sampling the detection image based on each Mamba block layer in the Backbone module, and performing multi-directional scanning processing on each sampling result, so that the robot can perform multi-directional scanning in a dynamic environment and overcome the change of the detection angle. In addition, the linear calculation complexity of Mamba has a low occupation of computing resources, thereby solving the shortcomings of low work efficiency, low robustness, low accuracy, low efficiency and large occupation of computing resources in the existing robot for target detection in a dynamic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a target detection method according to an embodiment of the present invention;

[0042] Figure 2 is a schematic diagram of the network structure of a deep learning model according to an embodiment of the present invention;

[0043] Figure 3 is a schematic diagram of the structure of the Backbone module of an embodiment of the present invention;

[0044] Figure 4 is a schematic diagram of the structure of the Mamba block layer in an embodiment of the present invention;

[0045] Figure 5 is a flow chart of a target detection method according to another embodiment of the present invention;

[0046] Figure 6 is a flow chart of a target detection method according to another embodiment of the present invention;

[0047] Figure 7 is a flow chart of a target detection method according to another embodiment of the present invention;

[0048] Figure 8 is a flow chart of a target detection method according to another embodiment of the present invention;

[0049] Fig. 9 is a schematic structural diagram of a target detection device according to an embodiment of the present invention;

[0050] Fig.10 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0051] Fig.11 It is a schematic diagram of the structure of a computer storage medium according to an embodiment of the present invention.

[0052] The meanings of the symbols in the accompanying drawings are:

[0053] 10-input end; 20-Backbone module; 30-Neck module; 40-Detect head module; 21-image block embedding layer; 22-layer normalization layer; 23-activation layer; 24-first Mamba block layer; 25-second Mamba block layer; 26-third Mamba block layer; 27-fourth Mamba block layer; 241-scanning unit; 242-information extraction unit; 243-information fusion unit; 244-layer normalization unit; 245-multilayer perceptron; 246-residual processing unit; 90-target detection device; 91-acquisition module; 92-detection module; 100-computer equipment; 101-processor; 102-memory; 111-program file. DETAILED DESCRIPTION

[0054] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0055] The terms "first", "second" and "third" in the present invention are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second" and "third" can explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. All directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0056] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0057] Figure 1 FIG. 1 is a flow chart of a target detection method according to an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present invention is not limited to Figure 1 The process sequence shown is limited. Figure 1 As shown, the method comprises the steps of:

[0058] Step S101: Acquire a detection image based on the shooting device of the robot.

[0059] In step S101, through advanced control methods, real-time remote control systems, end-to-end environmental perception technology, robot grasping technology and target detection algorithms, robots (such as humanoid machines) can achieve autonomous navigation, precise positioning and grasping of target objects in dynamic environments, thereby improving operating efficiency and safety in the fields of manufacturing, service industries and rescue, and promoting the development of automation and intelligence. Among them, the target detection algorithm enables the robot to better understand the surrounding environment, identify and track humans or other objects, thereby providing a more natural and intelligent interactive experience.

[0060] In the field of industrial manufacturing, robots usually need to perform multiple tasks, and are equipped with corresponding software and hardware devices for each task. As an example, when a robot needs to perform a visual task, it needs to be equipped with a camera to obtain images. For example, when the robot dynamically detects the environment, it uses a camera to obtain product image data at various angles and positions in the workshop, i.e., detection images.

[0061] Step S102: input the detection image into a pre-trained deep learning model for detection to obtain a detection result; the network structure of the deep learning model includes an input end, a Backbone module, a Neck module and a Detect head module connected in sequence, wherein the Backbone module includes a first Mamba block layer, a second Mamba block layer, a third Mamba block layer and a fourth Mamba block layer which are stacked and connected in sequence, and each Mamba block layer is connected to the Neck module respectively; the input end is used to input the detection image into the pre-trained deep learning model; each Mamba block layer in the Backbone module is used to downsample the detection image, and perform multi-directional scanning processing, channel transformation processing and size transformation processing on each sampling result, and output feature maps with different numbers of channels and different sizes; the Neck module is used to extract multi-scale features from the feature map; the Detect head module is used to predict according to the multi-scale features to obtain the detection result.

[0062] In step S102, in order to effectively capture global context information and help the model better understand the global distribution and mutual relationship of the entire image, the selective structured state space model Mamba is used as the backbone of the model framework to model the global context information and build a deep learning model to extract complex dependencies in sequence data, thereby focusing on the global context information. The SSM (State Space Model) of the discrete state in Mamba is expressed by the following formula:

[0063] y k =Ch k ; where x k is the input, y k is the output, h k represents the potential state at any given time k, h k―1 represents the previous potential state at time k, the matrix The hidden state matrix is ​​generated. C is the input x k Learnable parameters of the mapping.

[0064] The linear computational complexity of Mamba occupies less computing resources, models the complex dependencies of the sequence more fully, and makes the model more robust and has better performance. The deep learning model of the embodiment of the present invention utilizes the linear computational complexity of Mamba to well reduce the model's requirements for computing resources, achieve the purpose of lightweighting the model, and occupy less computing resources while maintaining excellent performance, thereby better supporting the robot to complete multiple tasks at the same time.

[0065] See also Figure 2The network structure of the deep learning model includes an input terminal 10, a Backbone module 20, a Neck module 30 and a Detect head module 40 connected in sequence, wherein the Backbone module 20 includes a first Mamba block layer 24, a second Mamba block layer 25, a third Mamba block layer 26 and a fourth Mamba block layer 27 which are stacked and connected in sequence, and the outputs of each Mamba block layer are respectively connected to the Neck module 30. When detecting a target, the input terminal 10 is first used to input the detection image into the pre-trained deep learning model; then each Mamba block layer in the Backbone module 20 downsamples the detection image, and performs multi-directional scanning processing, channel transformation processing and size transformation processing on each sampling result, and outputs feature maps with different channel numbers and different sizes; then the Neck module 30 extracts multi-scale features from the feature map; finally, the Detect head module 40 performs prediction based on the multi-scale features to obtain the detection result. Specifically, in the Backbone module 20, the first Mamba block layer 24 downsamples the detection image, performs multi-directional scanning processing, channel transformation processing and size transformation processing on the sampling result, and outputs the first feature map to the Neck module 30 and the second Mamba block layer 25; the second Mamba block layer 25 downsamples the first feature map output by the first Mamba block layer 24, performs multi-directional scanning processing, channel transformation processing and size transformation processing on the sampling result, and outputs the second feature map to the Neck module 30 and the third Mamba block layer 26; the third Mamba block layer 26 downsamples the second feature map output by the second Mamba block layer 25, performs multi-directional scanning processing, channel transformation processing and size transformation processing on the sampling result, and outputs the third feature map to the Neck module 30 and the fourth Mamba block layer 27; the fourth Mambablock layer 27 downsamples the third feature map output by the third Mamba block layer 26, performs multi-directional scanning processing, channel transformation processing and size transformation processing on the sampling result, and outputs the fourth feature map to the Neck module 30.Exemplarily, the input detection image (B, H, W, C) becomes (B, H / 2, W / 2, 2C) after the first Mamba Block layer 24, becomes (B, H / 4, W / 4, 4C) after the second Mamba Block layer 25, becomes (B, H / 8, W / 4, 8C) after the third Mamba Block layer 26, and becomes (B, H / 16, W / 16, 16C) after the fourth Mamba Block layer 27, that is, each Mamba Block layer outputs feature maps with different numbers of channels and different sizes to the Neck module 30, and the Neck module 30 uses a feature pyramid network to extract multi-scale features from the feature map and outputs them to the Detect head module 40. Wherein, B is the batch size, H is the height, W is the width, and C is the number of channels.

[0066] As an example, see Figure 3 The Backbone module 20 also includes an image block embedding layer 21, a layer normalization layer 22 and an activation layer 23 connected in sequence, wherein the output of the activation layer 23 is used as the input of the first Mamba block layer 24. The image block embedding layer 21 is used to perform block processing and image conversion processing on the detection image and output it; the layer normalization layer 22 is used to perform data normalization processing on the output of the image block embedding layer 21 and output it; the activation layer 23 is used to perform nonlinear transformation processing on the output of the layer normalization layer 22 and output it.

[0067] As an example, see Figure 4 Each Mamba block layer further includes: a plurality of scanning units 241, an information extraction unit 242 corresponding to the scanning unit 241, an information fusion unit 243 connected to the information extraction unit 242, a layer normalization unit 244 connected to the information fusion unit 243, a multi-layer perceptron 245 connected to the layer normalization unit 244, and a residual processing unit 246. The scanning directions of the plurality of scanning units 241 are different. The number of scanning units 241 set in each Mamba block layer can be the same or different. For example, see Figure 4 Taking the first Mamba block layer 24 as an example, three scanning units 241 are provided, and three information extraction units 242 are provided accordingly. The embodiment of the present invention realizes scanning in multiple scanning directions by providing multiple scanning units 241, enhances the robustness of the model in a dynamic environment, overcomes the change of the detection angle, and thus improves the robustness, accuracy and efficiency of the robot in detecting the target in a dynamic environment.

[0068] Before using the constructed deep learning model for target detection, the deep learning model deep learning model needs to be pre-trained. This embodiment uses an adaptive stride algorithm to increase the training bias for difficult-to-detect samples, enhance the training effect, and can adapt to the robot's target detection environment. Unfavorable training conditions such as large occlusion, illumination changes, target deformation and scale changes. The stride algorithm divides the sample image into M*N grids, and the grid size is set to the average defect size to increase the probability of the target center falling in the grid and being detected. Specifically, the average size (w, h) of the three target boxes is obtained by k-means clustering. If w is greater than h, the stride is set to w, otherwise, the stride is set to h. Where w is width and h is height.

[0069] As an example, see Figure 5 Before the step of downsampling the detection image at the first Mamba block layer, performing multi-directional scanning processing, channel transformation processing and size transformation processing on each sampling result, and outputting feature maps with different numbers of channels and different sizes, the following steps are also included:

[0070] Step S201: using the image block embedding layer to perform block processing and image conversion processing on the detection image and output it.

[0071] Specifically, the image conversion process may be a channel expansion process. Exemplarily, the detection image of (B, C, H, W) is processed by an image block embedding layer and output as (B, H, W, C).

[0072] Step S202: using a layer normalization layer to perform data normalization processing on the output of the image block embedding layer and output it.

[0073] Specifically, the layer normalization layer can make the data distribution closer to the normal distribution and improve the training efficiency and stability of the model.

[0074] Step S203: Use the activation layer to perform nonlinear transformation on the output of the layer normalization layer and output it.

[0075] Specifically, the activation layer uses the GeLu function to perform nonlinear transformation processing on the output of the normalization layer and outputs it.

[0076] As an example, see Figure 6 Each Mamba block layer performs multi-directional scanning processing, channel transformation processing and size transformation processing on each sampling result, and outputs feature maps with different numbers of channels and different sizes, and also includes the following steps:

[0077] Step S301: performing multi-directional scanning processing, channel transformation processing and size transformation processing on the sampling results through multiple scanning units.

[0078] Specifically, the scanning directions of the plurality of scanning units are different, which can be understood as one scanning unit is set with one scanning direction. Exemplarily, the scanning direction includes but is not limited to forward, reverse, oblique, and the like.

[0079] Step S302: The information extraction unit rearranges the output of the corresponding scanning unit and extracts global feature information.

[0080] Specifically, the information extraction unit and the scanning unit are set correspondingly, and the information extraction unit performs convolution operation and activation operation on the output of the corresponding scanning unit to obtain latent state space features; based on the preset selective structured state space model (SSM), the global context information of the latent state space features is selectively processed to obtain global feature information.

[0081] Step S303: The global feature information output by each information extraction unit is fused through the information fusion unit.

[0082] Specifically, the fusion method adopted by the information fusion unit may be spatial domain fusion, such as alpha fusion.

[0083] Step S304: normalize the output of the information fusion unit through the layer normalization unit.

[0084] Step S305: Introduce nonlinear changes to the output of the layer normalization unit through a multi-layer perceptron.

[0085] Step S306: Perform residual processing on the output of the information fusion unit and the output of the multi-layer perceptron through the residual processing unit.

[0086] As an example, see Figure 7 , after step S101 and before step S102, further comprising the following steps:

[0087] Step S100: preprocessing the detected image; the preprocessing is image enhancement processing.

[0088] In step S100, grayscale change and histogram equalization are used to perform image enhancement.

[0089] As an example, see Figure 8 , after step S102, further comprising the following steps:

[0090] Step S103: Post-processing the detection results. The post-processing is to determine whether to output the detection results based on the preset thresholds of each category.

[0091] In step S103, for the model output target, it is determined whether to output the detection result based on the preset threshold of each category, and excludes the target located in the abnormal area, so as to improve the accuracy of target detection. In this embodiment, the detection result is the upper left corner coordinates, height and width of all targets.

[0092] Fig. 9 Schematic diagram of the structure of the target detection device according to an embodiment of the present invention. Fig. 9 As shown, the target detection device 90 includes an acquisition module 91 and a detection module 92 .

[0093] The acquisition module 91 is used to acquire the detection image based on the shooting device of the robot;

[0094] The detection module 92 is used to input the detection image into the pre-trained deep learning model for detection to obtain the detection result; the network structure of the deep learning model includes an input end, a Backbone module, a Neck module and a Detect head module connected in sequence, wherein the Backbone module includes a first Mamba block layer, a second Mamba block layer, a third Mamba block layer and a fourth Mamba block layer which are stacked and connected in sequence, and each Mamba block layer is connected to the Neck module respectively;

[0095] The input end is used to input the detection image into the pre-trained deep learning model;

[0096] Each Mamba block layer in the Backbone module is used to downsample the detection image, and perform multi-directional scanning, channel transformation and size transformation on each sampling result, and output feature maps with different numbers of channels and different sizes;

[0097] The Neck module is used to extract multi-scale features from the feature map;

[0098] The Detect head module is used to make predictions based on multi-scale features and obtain detection results.

[0099] See also Fig.10 , Fig.10 FIG. 1 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Fig.10 As shown, the computer device 100 includes a processor 101 and a memory 102 coupled to the processor 101 .

[0100] The memory 102 stores program instructions for implementing the target detection method described in any of the above embodiments.

[0101] The processor 101 is used to execute program instructions stored in the memory 102 to detect a target.

[0102] The processor 101 may also be referred to as a CPU (Central Processing Unit). The processor 101 may be an integrated circuit chip having signal processing capabilities. The processor 101 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0103] See also Fig.11 , Fig.11 Schematic diagram of the structure of the computer storage medium of the embodiment of the present invention. The computer storage medium of the embodiment of the present invention stores a program file 111 that can implement all the above methods, wherein the program file 111 can be stored in the above computer storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned computer storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes, or terminal devices such as computers, servers, mobile phones, tablets, etc.

[0104] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0105] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0106] The above are only implementation modes of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A target detection method, characterized in that: include: Acquire detection images based on the robot’s shooting equipment; Inputting the detection image into a pre-trained deep learning model for detection to obtain a detection result; The network structure of the deep learning model includes an input end, a Backbone module, a Neck module and a Detect head module connected in sequence, wherein the Backbone module includes a first Mamba block layer, a second Mambablock layer, a third Mamba block layer and a fourth Mamba block layer which are stacked and connected in sequence, and each Mamba block layer is connected to the Neck module respectively; The input end is used to input the detection image into a pre-trained deep learning model; Each Mamba block layer in the Backbone module is used to downsample the detection image, and perform multi-directional scanning processing, channel transformation processing and size transformation processing on each sampling result, and output feature maps with different numbers of channels and different sizes; The Neck module is used to extract multi-scale features from the feature map; The Detect head module is used to make predictions based on the multi-scale features to obtain the detection results.

2. The target detection method according to claim 1, characterized in that: The Backbone module further includes an image block embedding layer, a layer standardization layer, and an activation layer; before the first Mamba block layer performs the steps of downsampling the detection image, performing multi-directional scanning processing, channel transformation processing, and size transformation processing on each sampling result, and outputting feature maps with different numbers of channels and different sizes, the following steps are also included: Using the image block embedding layer to perform block processing and image conversion processing on the detection image and output it; Using the layer normalization layer to perform data normalization processing on the output of the image block embedding layer and output it; The activation layer is used to perform nonlinear transformation processing on the output of the normalization layer and output the resultant output.

3. The target detection method according to claim 1, characterized in that: Each Mamba block layer also includes: multiple scanning units, information extraction units corresponding to the scanning units, information fusion units connected to the information extraction units, layer standardization units connected to the information fusion units, multi-layer perceptrons connected to the layer standardization units, and residual processing units, wherein the scanning directions of the multiple scanning units are different.

4. The target detection method according to claim 3, characterized in that: Each Mamba block layer performs the steps of performing multi-directional scanning processing, channel transformation processing, and size transformation processing on each sampling result, and outputting feature maps with different numbers of channels and different sizes, and also includes the following steps: Perform multi-directional scanning processing, channel transformation processing, and size transformation processing on the sampling results through a plurality of the scanning units; The information extraction unit rearranges the output of the corresponding scanning unit and extracts the global feature information; fusing the global feature information output by each of the information extraction units through the information fusion unit; Normalizing the output of the information fusion unit by the layer normalization unit; Introducing nonlinear changes to the output of the layer normalization unit through the multilayer perceptron; The residual processing unit performs residual processing on the output of the information fusion unit and the output of the multi-layer perceptron.

5. The target detection method according to claim 4, characterized in that: The step of rearranging the outputs of the corresponding scanning units and extracting the global feature information by the information extraction unit comprises the following steps: Perform convolution and activation operations on the output of the corresponding scanning unit to obtain the latent state space features; The global context information of the latent state space feature is selectively processed based on a preset selective structured state space model to obtain the global feature information.

6. The target detection method according to claim 1, characterized in that: After the robot-based shooting device acquires the detection image and before the detection image is input into the pre-trained deep learning model for detection, the following steps are also included: The detection image is preprocessed; the preprocessing is image enhancement processing.

7. The target detection method according to claim 1, characterized in that: After obtaining the test results, the following steps are also included: The detection result is post-processed, and the post-processing is to determine whether to output the detection result based on the preset threshold value of each category.

8. A target detection device, characterized in that: Applied to the target detection method according to any one of claims 1 to 7, the target detection device comprises: An acquisition module, used for acquiring detection images based on a shooting device of the robot; A detection module is used to input the detection image into a pre-trained deep learning model for detection to obtain a detection result; the network structure of the deep learning model includes an input end, a Backbone module, a Neck module and a Detect head module connected in sequence, wherein the Backbone module includes a first Mambablock layer, a second Mamba block layer, a third Mamba block layer and a fourth Mamba block layer which are stacked and connected in sequence, and each Mambablock layer is connected to the Neck module respectively; The input end is used to input the detection image into a pre-trained deep learning model; Each Mamba block layer in the Backbone module is used to downsample the detection image, and perform multi-directional scanning processing, channel transformation processing and size transformation processing on each sampling result, and output feature maps with different numbers of channels and different sizes; The Neck module is used to extract multi-scale features from the feature map; The Detect head module is used to make predictions based on the multi-scale features to obtain the detection results.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the target detection method according to any one of claims 1 to 7 when executing the computer program.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the target detection method according to any one of claims 1 to 7 is implemented.

Images