Detection model training method, pedestrian detection method, pedestrian detection device and related equipment

Abstract

The embodiment of the invention provides a training method of a detection model, a pedestrian detection method and device and related equipment, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring a sample pedestrian image and a sample background image matched with a scene in the sample pedestrian image, and inputting the sample pedestrian image and the sample background image into an initial detection model; feature extraction is carried out on the background in the sample background image to obtain initial background features, feature extraction is carried out on pedestrians in the sample pedestrian image to obtain initial pedestrian features, and the initial background features and the initial pedestrian features are fused to obtain sample fusion features; predicting a background in the sample fusion feature to obtain a sample background feature; and calculating a background loss value based on the sample background features and the initial background features, and training an initial detection model according to the background loss value to obtain a trained detection model. The accuracy of pedestrian detection can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a detection model training method, a pedestrian detection method, a device and related equipment. Background Art

[0002] Pedestrian Detection refers to the technology that uses computer vision technology to determine whether there are pedestrians in a given image or video sequence and other visual information, and to provide accurate positioning.

[0003] In the related art, the detection model is trained using images containing both pedestrians and their backgrounds, so that the trained detection model can identify pedestrians from the input target image. However, the model training method in the related art has high requirements for the data set, which requires the data set to cover more scenes, resulting in poor generalization ability of the detection model. Once the quality of the data set fails to meet the requirements, the quality of the trained detection model will be reduced, thereby reducing the accuracy of pedestrian detection. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a detection model training method, a pedestrian detection method, an apparatus and related equipment, aiming to improve the accuracy of pedestrian detection.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for training a detection model, the method comprising: Obtaining a sample pedestrian image and a sample background image matching a scene in the sample pedestrian image, and inputting the sample pedestrian image and the sample background image into an initial detection model; Extracting features of the background in the sample background image to obtain initial background features, extracting features of pedestrians in the sample pedestrian image to obtain initial pedestrian features, and fusing the initial background features and the initial pedestrian features to obtain sample fusion features; Predict the background in the sample fusion features to obtain the sample background features; A background loss value is calculated based on the sample background features and the initial background features, and an initial detection model is trained according to the background loss value to obtain a trained detection model.

[0006] In some embodiments, the initial detection model is trained according to the background loss value to obtain a trained detection model, including: Predict the pedestrians in the sample fusion features to obtain the sample pedestrian features; The pedestrian loss value is calculated based on the sample pedestrian features and the initial pedestrian features, and the initial detection model is trained according to the background loss value and the pedestrian loss value to obtain the trained detection model.

[0007] In some embodiments, the initial detection model includes a first feature extractor, a second feature extractor, a background detection network and a pedestrian detection network, the initial background feature is extracted by the first feature extractor, the initial pedestrian feature is extracted by the second feature extractor, the sample background feature is predicted by the background detection network, and the sample pedestrian feature is predicted by the pedestrian detection network; The initial detection model is trained according to the background loss value and the pedestrian loss value to obtain the trained detection model, including: The second parameters corresponding to the second feature extractor and the pedestrian detection network are fixed, and the first parameters corresponding to the first feature extractor and the background detection network are adjusted according to the background loss value; Fixing the first parameters corresponding to the first feature extractor and the background detection network, and adjusting the second parameters corresponding to the second feature extractor and the pedestrian detection network according to the pedestrian loss value; The first parameter and the second parameter are adjusted alternately until a preset training condition is reached, thereby obtaining a trained detection model.

[0008] In some embodiments, the first feature extractor includes a plurality of first convolutional layers connected in sequence, each first convolutional layer being used to perform a first convolution processing operation on the input image; Extract the background features in the sample background image to obtain the initial background features, including: Performing at least one first convolution operation on the sample background image, each first convolution layer outputs a corresponding first intermediate image, wherein, except for the first first convolution layer, the other first convolution layers perform the first convolution operation on the first intermediate image output by the previous first convolution layer; A feature extraction operation is performed on the first intermediate image output by the last first convolutional layer to obtain initial background features.

[0009] In some embodiments, the second feature extractor includes a plurality of second convolutional layers connected in sequence, each second convolutional layer being used to perform a second convolution processing operation on the input image; Extract the features of pedestrians in the sample pedestrian image and obtain the initial pedestrian features, including: Performing at least one second convolution operation on the sample pedestrian image, each second convolution layer outputs a corresponding second intermediate image, wherein, except for the first second convolution layer, the other second convolution layers perform the second convolution operation on the second intermediate image output by the previous second convolution layer; A feature extraction operation is performed on the second intermediate image output by the last second convolutional layer to obtain the initial pedestrian features.

[0010] In some embodiments, the initial detection model is trained according to the background loss value and the pedestrian loss value to obtain a trained detection model, further comprising: Get the real verification label corresponding to the sample pedestrian image; The sample pedestrian image is predicted based on the sample pedestrian features to obtain the sample prediction result, and the pedestrian loss value is calculated based on the real verification label and the sample prediction result; A background loss value is calculated based on the initial background features and the sample background features, and the difference between the background loss value and the pedestrian loss value is determined as the detection loss value; The initial detection model is trained according to the detection loss value to obtain a trained detection model.

[0011] In some embodiments, the real verification label includes real position information, real size information, and real category information, and the sample prediction result includes predicted position information, predicted size information, and predicted category information; The pedestrian loss value is calculated based on the real verification label and sample prediction results, including: Calculate a first square operation result of the difference between the actual position information and the predicted position information, and determine a multiplication value of a preset first weight and the first square operation result as a first loss term; Calculate the difference between the square root operation result of the real size information and the square root operation result of the predicted size information, and determine that the multiplication value between the preset second weight and the second square operation result of the result difference is a second loss term; Calculate a third square operation result of the difference between the predicted category information and the actual category information, and determine a third loss term as a multiplication value of a preset third weight and the third square operation result; The sum of the first loss item, the second loss item and the third loss item is determined to be the pedestrian loss value.

[0012] To achieve the above object, a second aspect of an embodiment of the present application proposes a pedestrian detection method, the method comprising: Obtaining a sample pedestrian image and a sample background image matching a scene in the sample pedestrian image, and inputting the sample pedestrian image and the sample background image into an initial detection model; Extracting features of the background in the sample background image to obtain initial background features, extracting features of pedestrians in the sample pedestrian image to obtain initial pedestrian features, and fusing the initial background features and the initial pedestrian features to obtain sample fusion features; Predict the background in the sample fusion features to obtain the sample background features; A background loss value is calculated based on the sample background features and the initial background features, and an initial detection model is trained according to the background loss value to obtain a trained detection model; Acquire a target pedestrian image and a target background image, wherein the target pedestrian image and the sample pedestrian image are different; The target pedestrian image and the target background image are input into the trained detection model to obtain the target detection result.

[0013] To achieve the above-mentioned purpose, a third aspect of an embodiment of the present application proposes a training device for a detection model, the device comprising: An acquisition module, used for acquiring a sample pedestrian image and a sample background image matching the scene in the sample pedestrian image, and inputting the sample pedestrian image and the sample background image into an initial detection model; A feature extraction module is used to extract features of the background in the sample background image to obtain initial background features, and to extract features of pedestrians in the sample pedestrian image to obtain initial pedestrian features, and to fuse the initial background features and the initial pedestrian features to obtain sample fusion features; A prediction module is used to predict the background in the sample fusion feature to obtain the sample background feature; The training module is used to calculate the background loss value based on the sample background features and the initial background features, train the initial detection model according to the background loss value, and obtain the trained detection model.

[0014] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the training method of the detection model of the first aspect or the pedestrian detection method of the second aspect.

[0015] To achieve the above-mentioned purpose, the fifth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the training method of the detection model of the first aspect or the pedestrian detection method of the second aspect.

[0016] The present application proposes a detection model training method, a pedestrian detection method, an apparatus and related equipment, which obtain a sample pedestrian image and a sample background image that matches the scene in the sample pedestrian image, and input the sample pedestrian image and the sample background image into an initial detection model; extract features of the background in the sample background image to obtain initial background features, and extract features of the pedestrian in the sample pedestrian image to obtain initial pedestrian features, and fuse the initial background features and the initial pedestrian features to obtain sample fusion features; the initial background features and the initial pedestrian features contain different information, based on which the initial detection model separates the features of the pedestrian and the features of the background, so as to further learn the ability to distinguish between background features and pedestrian features from the sample fusion features mixed with the background and pedestrians after targeted learning of the features of the background and pedestrians respectively; then, predict the background in the sample fusion features to obtain sample background features; calculate a background loss value based on the sample background features and the initial background features, and train the initial detection model according to the background loss value to obtain a trained detection model. The trained detection model can identify the background from the input image, distinguish the background from pedestrians, and perform pedestrian detection operations, thereby improving the accuracy of pedestrian detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of an optional application scenario of the training device for the detection model provided in the embodiment of the present application; Figure 2 It is an optional flowchart of the training method of the detection model provided in the embodiment of the present application; Figure 3 It is an optional sample image schematic diagram of the training method of the detection model provided in the embodiment of the present application; Figure 4 yes Figure 2 A flowchart of an implementation of step 102 in FIG. Figure 5 It is a schematic diagram of the structure and application of an optional first feature extractor of the detection model training method provided in the embodiment of the present application; Figure 6 yes Figure 2 Another implementation flowchart of step 102 in FIG. Figure 7 It is a schematic diagram of the structure and application of an optional second feature extractor of the detection model training method provided in the embodiment of the present application; Figure 8 yes Figure 2 A flowchart of an implementation of step 104 in FIG. Fig. 9 yes Figure 8 A flowchart of an implementation of step 402 in FIG. Fig.10 It is another optional flowchart of the training method of the detection model provided in the embodiment of the present application; Fig.11 yes Figure 8 Another implementation flowchart of step 402 in FIG. Fig.12 yes Fig.11 A flowchart of an implementation of step 602 in FIG. Fig.13 is an optional flow chart of the pedestrian detection method provided in the embodiment of the present application; Fig.14 It is a schematic diagram of an optional functional module of the training device of the detection model provided in the embodiment of the present application; Fig.15 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0021] First, some nouns involved in this application are analyzed: Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0022] The L1 norm, also known as the Manhattan distance or the absolute value norm, refers to the sum of the absolute values ​​of each element in a vector.

[0023] Pedestrian detection refers to the technology that uses computer vision technology to determine whether there are pedestrians in a given image or video sequence and other visual information, and to provide precise positioning.

[0024] In the related art, the detection model is trained using images containing both pedestrians and their backgrounds, so that the trained detection model can identify pedestrians from the input target image. However, the model training method in the related art has high requirements for the data set, which requires the data set to cover more scenes, resulting in poor generalization ability of the detection model. Once the quality of the data set fails to meet the requirements, the quality of the trained detection model will be reduced, thereby reducing the accuracy of pedestrian detection.

[0025] Based on this, the embodiments of the present application provide a detection model training method, a pedestrian detection method, an apparatus and related equipment, aiming to improve the accuracy of pedestrian detection.

[0026] The detection model training method, pedestrian detection method, device and related equipment provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the detection model training method in the embodiments of the present application is described.

[0027] It should be noted that in the embodiments of the present application, when it comes to information related to user characteristics such as user basic information or user identity, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the user's separate permission or consent will be obtained first. After the user's separate permission or consent is clearly obtained, the necessary data for the normal operation of the embodiments of the present application will be obtained. For example, when the embodiments of the present application obtain sample pedestrian images and sample background images taken by the user, the consent of the relevant user will be obtained first; or, if the obtained sample pedestrian images and sample background images involve privacy including but not limited to the user's portrait privacy, personal information, etc., the consent of the relevant user will be obtained first, otherwise the sample pedestrian images and sample background images used in the embodiments of the present application cannot be obtained.

[0028] The detection model training method, pedestrian detection method, device and related equipment provided in the embodiments of the present application are specifically described through the following embodiments. First, the application scenario of the detection model training device in the embodiments of the present application (for the convenience of description, it may also be referred to as "training device" below) is described, such as Figure 1 As shown, Figure 1 This is a schematic diagram of an optional application scenario of a training device for a detection model provided in an embodiment of the present application. First, the acquired sample pedestrian images and sample background images are input into the training device provided in an embodiment of the present application. The training device can extract features from the sample pedestrian images and the sample background images respectively and fuse them to obtain sample fusion features; predict the background in the sample fusion features to obtain sample background features; calculate the background loss value based on the sample background features, and adjust the parameters of the detection model based on the background loss value to obtain a trained detection model; the ability of the trained detection model in background recognition detection of images can be significantly improved, thereby improving the accuracy of its overall image detection.

[0029] After understanding the application scenarios of the training device proposed in the present application, the training method of the detection model proposed in the embodiment of the present application is explained next.

[0030] In the embodiments of the present application, the training device will be described from the perspective of a training device, which can be integrated into a computer device, such as a server. Figure 2 As shown, Figure 2 is an optional flowchart of the method for training the detection model provided in the embodiment of the present application. Figure 2The method may include but is not limited to the following steps 101 to 104. When the training device executes the training method of the detection model, the specific process is as follows. It should be noted that the present embodiment Figure 2 The order of step 101 to step 104 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0031] Step 101: obtain a sample pedestrian image and a sample background image that matches the scene in the sample pedestrian image, and input the sample pedestrian image and the sample background image into an initial detection model.

[0032] The following is a detailed description of step 101.

[0033] like Figure 3 As shown, Figure 3 This is an optional sample image schematic diagram of the training method of the detection model provided in the embodiment of the present application, wherein the sample pedestrian image refers to an image containing pedestrians, and pedestrians account for a large proportion of the sample pedestrian image, so that the initial detection model can better extract the initial pedestrian features and perform subsequent processing and learning based on the extracted features. In addition, the sample pedestrian image includes not only pedestrians but also redundant background information; specifically, in order to completely capture pedestrians, the sample pedestrian image will inevitably contain unnecessary background, which is called "redundant background information".

[0034] The sample background image refers to an image that does not contain pedestrians but only contains pure background. The scene of the sample background image matches the scene in the sample pedestrian image, that is, the sample background image matches the redundant background information in the sample pedestrian image. For example, Figure 3 As shown, sample pedestrian image 2 (can be Figure 3 If any of the three sample pedestrian images 2 in the training data shows pedestrians walking in a shopping mall, then the scene in the sample background image 2 input into the initial detection model together with the sample pedestrian image 2 is also a shopping mall, and the sample background image does not contain any pedestrians. Sample background images containing pure backgrounds help the model better learn and distinguish the difference between background and pedestrians, thereby enhancing the trained detection model's ability to recognize pedestrians.

[0035] Furthermore, the sample pedestrian image can be obtained by real-time photography by a camera; or, it can be obtained from an open pedestrian image database, which can be the Caltech Pedestrian Detection Benchmark (Caltech Pedestrian Database), the University of Southern California Pedestrian Detection Test Set (USC Pedestrian Detection Test Set, USC Pedestrian Database), or the Daimler Pedestrian Detection Benchmark (Daimler Pedestrian Database); or, the acquired image is processed to obtain a sample pedestrian image, for example, a pedestrian image to be processed is randomly selected from the Caltech Pedestrian Database, and a sample pedestrian image containing a pedestrian is cropped out from it, or, the sample background image and the sample pedestrian image are captured from a video. The embodiment of the present application does not limit the method for obtaining the sample pedestrian image, and can be adaptively adjusted according to actual conditions.

[0036] Furthermore, the sample background image can be obtained by real-time camera shooting; or, it can be obtained from an open background image database; or, the obtained image is processed to obtain the sample background image, for example, a background image to be processed is randomly selected from the background image database, and the edge of the background image contains pedestrians, then the pedestrians are cut out from the background image to be processed, and then the sample background image is obtained. The embodiment of the present application does not limit the method for obtaining the sample background image, and it can be adaptively adjusted according to actual conditions.

[0037] Furthermore, the scenes in the sample background image and the sample pedestrian image input to the initial detection model may not match. In this way, the initial detection model is helped to perform adaptive learning by randomly matching the sample pedestrian image and the sample background image. The initial detection model can learn the similarities between the sample background images with the same scene as the sample pedestrian image and the differences between the sample background images with different scenes from the sample pedestrian image during the training process, thereby helping the trained detection model to adapt faster and make accurate judgments when facing new and unseen backgrounds, thereby improving its generalization ability.

[0038] Furthermore, during the training process, the same sample pedestrian image can correspond to multiple sample background images, that is, the initial pedestrian feature can be fused with a single initial background feature, or it can be fused with multiple initial background features to obtain a sample fusion feature. The embodiment of the present application does not limit the number of sample background images corresponding to a sample pedestrian image, and can be set according to actual conditions. In this way, the embodiment of the present application can combine multiple sample fusion features by performing feature extraction and fusion operations on sample background images of the same type of scene or different types of scenes at different angles and different lighting conditions, thereby helping to improve the generalization ability of the initial detection model.

[0039] Step 102, extracting features from the background in the sample background image to obtain initial background features, extracting features from pedestrians in the sample pedestrian image to obtain initial pedestrian features, and fusing the initial background features and initial pedestrian features to obtain sample fusion features.

[0040] The following is a detailed description of step 102.

[0041] In some embodiments, after the sample background image and the sample pedestrian image are input into the initial detection model, the initial detection model will perform feature extraction operations on the sample background image and the sample pedestrian image respectively to obtain initial background features and initial pedestrian features. It can be understood that the initial background features and the initial pedestrian features contain different information respectively. The embodiment of the present application separates the features of the pedestrian and the features of the background, so that the initial detection model can learn the features of the background and the pedestrian respectively in a targeted manner.

[0042] Furthermore, after the initial detection model learns the initial pedestrian features and the initial background features respectively, the initial background features and the initial pedestrian features are fused to obtain sample fusion features, so that the subsequent initial detection model can predict the background and pedestrians in the sample fusion features. The fused sample fusion features can be a feature map, that is, an image containing background features and pedestrian features.

[0043] Furthermore, during the training process of the initial detection model, a sample pedestrian image and a sample background image may be used as a training group, that is, the initial detection model extracts a single initial pedestrian feature from a single sample pedestrian image, and extracts a single initial background feature from a single sample background image, and fuses the single initial pedestrian feature with the single initial background feature to obtain a sample fusion feature; or, during the training process of the initial detection model, a sample pedestrian image and multiple sample background images may be used as a training group, that is, the initial detection model extracts a single initial pedestrian feature from a single sample pedestrian image, and extracts multiple initial background features from multiple sample background images, and fuses the single initial pedestrian feature with each initial background feature to obtain multiple sample fusion features. In the latter case, the initial pedestrian feature and the initial background feature can be combined to obtain a variety of different sample fusion features to improve the generalization ability of the final detection model for unknown input images.

[0044] It is understandable that the target pedestrian image to be detected input into the detection model during actual application is usually a mixed image of background and pedestrians. That is, after the initial detection model of the present application completes the independent learning of background-related features and pedestrian-related features, it is still necessary to predict and process the sample fusion features that are integrated with background features and pedestrian features, so that the initial detection model can better distinguish the background part and the pedestrian part from the sample fusion features on the basis of independently detecting and identifying the characteristics of the background and pedestrians. For example, the related technology needs to perform pedestrian detection processing on an image to be detected that includes pedestrians, and the background part of the image to be detected includes a clothing display area. Since pedestrians also wear clothes and the model used in the related technology is trained based on a data set that contains both background and pedestrians, when the data set is insufficient or the quality is not high, the related technology is very likely to mistakenly detect a piece of displayed clothing as a pedestrian in the output result of the image to be detected; when training with a data set of the same quality, the embodiment of the present application can, on the basis of targeted extraction and learning of pedestrian features and background features respectively, predict the background and pedestrians in the sample fusion features by comparing the differences between the two. In this way, the pedestrian detection capability of the trained detection model can be improved.

[0045] In some embodiments, Figure 4 As shown, Figure 4 yes Figure 2 A flowchart for implementing step 102 in the embodiment of the present invention is provided, wherein the feature extraction of the background in the sample background image is performed to obtain the initial background feature, including the following steps 201 to 202: Step 201, performing at least one first convolution processing operation on the sample background image, each first convolution layer outputs a corresponding first intermediate image, wherein, except for the first first convolution layer, other first convolution layers all perform the first convolution processing operation on the first intermediate image output by the previous first convolution layer.

[0046] Step 202: Perform a feature extraction operation on the first intermediate image output by the last first convolutional layer to obtain initial background features.

[0047] Steps 201 to 202 are described in detail below.

[0048] Among them, the first feature extractor includes a plurality of first convolutional layers connected in sequence, and each first convolutional layer is used to perform a first convolution processing operation on the input sample background image. The first feature extractor can extract features related to the "background" from the sample background image. For example, features of trees, features of chairs, etc. are all features related to the "background". It should be noted that the "first" mentioned in steps 201 to 202 is only used for distinction. For example, the first convolutional layer in steps 201 to 202 is different from the second convolutional layer in steps 301 to 302.

[0049] For example, Figure 5 As shown, Figure 5 It is a schematic diagram of the structure and application of an optional first feature extractor of the training method of the detection model provided in an embodiment of the present application, wherein the first feature extractor includes a first first convolutional layer (convolutional layer 1), a second first convolutional layer (convolutional layer 2), and a third first convolutional layer (convolutional layer 3) connected in sequence, and the convolution kernel sizes of each first convolutional layer are 112*112*64, 28*28*128, and 7*7*1024, respectively. Convolutional layer 1 takes a sample background image as input, performs a convolution operation on the sample background image, and outputs The first intermediate image a; then, the convolution layer 2 takes the first intermediate image a as input, performs a convolution operation on the first intermediate image a, and outputs a first intermediate image b, wherein the convolution layer 2 is connected to the convolution layer 1; then, the convolution layer 3 takes the first intermediate image b as input, performs a convolution operation on the first intermediate image b, and outputs a first intermediate image c, wherein the convolution layer 3 is connected to the convolution layer 2, and the convolution layer 3 is the last first convolution layer; finally, a feature extraction operation is performed on the first intermediate image c to obtain the initial background features corresponding to the sample background image.

[0050] Among them, the convolution kernel is an adjustable matrix or array, which is used to perform convolution operations on input data; specifically, the convolution kernel is used to perform pixel-by-pixel weighted sum operations with the input image to generate an output image or feature map.

[0051] It should be noted that the number of first convolutional layers included in the first feature extractor and the size of the convolution kernel of each first convolutional layer can be set according to actual conditions, and the embodiment of the present application does not limit this. In addition, each first convolutional layer can also be provided with a first pooling layer, a first activation layer and other processing layers, which can be set according to actual conditions.

[0052] Furthermore, the first feature extractor can be composed of structures such as a convolutional neural network (CNN), a deep neural network (DNN) or an attention mechanism network (Attention Mechanism), and can be set specifically according to actual conditions, and the embodiments of the present application do not impose any restrictions on this.

[0053] Furthermore, in order to eliminate or reduce the noise in the initial background features and help the initial detection model to better learn the information related to the background, after obtaining the initial background features, the embodiment of the present application can also perform preprocessing operations on the initial background features to improve the data quality of the initial background features. Among them, the preprocessing operations include but are not limited to data cleaning, data conversion, random cropping, random flipping and random erasing, etc. The specific preprocessing operations and the order of operations between them can be set according to the actual situation, and the embodiment of the present application does not limit this.

[0054] It can be understood that, for sample background images under the same type of scene, the initial detection model can learn the similarities of each sample background image under different angles and lighting conditions based on the initial background images obtained by feature extraction from different sample background images; for sample background images under different types of scenes, the initial detection model can learn the differences between each sample background image based on the initial background images obtained by feature extraction from different sample background images; in either case, the extracted initial background features are only related to the background, based on this, the initial detection model can specifically learn features related to the background, so as to help the final trained detection model to recognize the "background".

[0055] In some embodiments, Figure 6 As shown, Figure 6 yes Figure 2 Another implementation flow chart of step 102 in the embodiment of the present invention is to extract features of pedestrians in the sample pedestrian image to obtain initial pedestrian features, including the following steps 301 to 302: Step 301, performing at least one second convolution processing operation on the sample pedestrian image, each second convolution layer outputs a corresponding second intermediate image, wherein, except for the first second convolution layer, the other second convolution layers all perform the second convolution processing operation on the second intermediate image output by the previous second convolution layer.

[0056] Step 302: Perform a feature extraction operation on the second intermediate image output by the last second convolutional layer to obtain initial pedestrian features.

[0057] Steps 301 to 302 are described in detail below.

[0058] The second feature extractor includes a plurality of second convolutional layers connected in sequence, and each second convolutional layer is used to perform a second convolution processing operation on the input sample pedestrian image. The second feature extractor can extract features related to "pedestrians" from the sample pedestrian image. For example, features of standing people, features of sitting people, and features of cyclists are all features related to "pedestrians". It should be noted that the "second" mentioned in steps 301 to 302 is only used for distinction.

[0059] For example, Figure 7 As shown, Figure 7 It is a schematic diagram of the structure and application of an optional second feature extractor of the training method of the detection model provided in an embodiment of the present application, wherein the second feature extractor includes a first second convolutional layer (convolutional layer 4), a second second convolutional layer (convolutional layer 5), a third second convolutional layer (convolutional layer 6), a fourth second convolutional layer (convolutional layer 7), and a fifth second convolutional layer (convolutional layer 8) connected in sequence, and the convolution kernel sizes of each second convolutional layer are 3*3*192, 3*3*256, 3*3*512, 3*3*1024, and 3*3*102 respectively. 4, convolution layer 4 takes the sample pedestrian image as input, performs a convolution operation on the sample pedestrian image, and outputs a second intermediate image d; then, convolution layer 5 takes the second intermediate image d as input, performs a convolution operation on the second intermediate image d, and outputs a second intermediate image e, wherein convolution layer 5 is connected to convolution layer 4; the input and output of convolution layer 6 to convolution layer 8 are similar to those of convolution layer 5, which will not be repeated here; finally, when convolution layer 8 outputs the second intermediate image f, a feature extraction operation is performed on the second intermediate image f to obtain the initial pedestrian features corresponding to the sample pedestrian image.

[0060] It should be noted that the number of second convolutional layers included in the second feature extractor and the size of the convolution kernel of each second convolutional layer can be set according to actual conditions, and the embodiment of the present application does not limit this. In addition, each second convolutional layer can also be provided with a second pooling layer, a second activation layer and other processing layers, which can be set according to actual conditions. Furthermore, the second feature extractor can also be composed of structures such as CNN and DNN, and the embodiment of the present application does not limit this.

[0061] Furthermore, in order to eliminate or reduce the noise in the initial pedestrian features and help the initial detection model better learn information related to pedestrians, after obtaining the initial pedestrian features, the embodiment of the present application can perform preprocessing operations on the initial pedestrian features to improve the data quality of the initial pedestrian features. Among them, the preprocessing operations include but are not limited to data cleaning, data conversion, random cropping, random flipping and random erasing, etc. The specific preprocessing operations and the order of operations between them can be set according to actual conditions, and the embodiment of the present application does not limit this.

[0062] It can be understood that the initial detection model can specifically learn the features related to pedestrians extracted by the second feature extractor to help train the detection model's ability to recognize "pedestrians".

[0063] Step 103: predict the background in the sample fusion feature to obtain the sample background feature.

[0064] The following is a detailed description of step 103.

[0065] In some embodiments, after obtaining the sample fusion feature, the initial detection model will predict the background in the sample fusion feature and obtain the sample background feature. The sample background feature is used to calculate the loss value used to help adjust the model parameters with the initial background feature extracted previously.

[0066] Step 104, a background loss value is calculated based on the sample background features and the initial background features, and an initial detection model is trained according to the background loss value to obtain a trained detection model.

[0067] Step 104 is described in detail below.

[0068] Among them, the background loss value reflects the difference between the background-related features extracted by the initial detection model from the sample background image and the background-related features obtained by distinguishing pedestrian features from the sample fusion features; the smaller the background loss value, the more similar the features extracted by the two are. After the background loss value is calculated, the model parameters of the initial detection model are adjusted according to the background loss value to obtain the trained detection model. The trained detection model can identify the background from the input image as the goal, distinguish the background and pedestrians, and perform pedestrian detection operations, thereby improving the accuracy of pedestrian detection.

[0069] Among them, the model parameters can be parameters such as the number of convolutional layers, the convolution kernel size of each convolutional layer, the batch size, the learning rate, etc. The model parameters that need to be adjusted can be adaptively adjusted according to actual conditions, and the embodiments of the present application do not limit this.

[0070] Furthermore, the operation steps of processing the sample pedestrian image and the sample background image described in step 102 to step 104 will be iteratively executed until the preset training conditions are met, and the trained detection model is obtained. The preset training conditions may be reaching a preset number of training times, or reaching a preset training time, or the background loss value may be less than a preset threshold; of course, the training conditions may be specifically set according to the actual situation, and the embodiment of the present application does not limit this.

[0071] It is understandable that the cluttered background is the biggest interference factor in pedestrian detection. On the basis of the initial detection model's targeted learning of the background recognition capability, the background in the sample fusion features that integrate pedestrian features and background features is predicted in order to better distinguish "pedestrians" from "background".

[0072] In some embodiments, Figure 8 As shown, Figure 8 yes Figure 2 A flowchart for implementing step 104 in the embodiment of the present invention is provided, wherein the initial detection model is trained according to the background loss value to obtain the trained detection model, including the following steps 401 to 402: Step 401, predicting pedestrians in sample fusion features to obtain sample pedestrian features.

[0073] Step 402: A pedestrian loss value is calculated based on the sample pedestrian features and the initial pedestrian features, and an initial detection model is trained according to the background loss value and the pedestrian loss value to obtain a trained detection model.

[0074] Steps 401 to 402 are described in detail below.

[0075] In some embodiments, the initial detection model can predict not only the background in the sample fusion features, but also the pedestrians in the sample fusion features, thereby obtaining the sample pedestrian features in the sample fusion features that indicate the pedestrians, so that the pedestrian loss value can be calculated based on the sample pedestrian features, and the model parameters of the initial detection model can be jointly adjusted based on the background loss value and the pedestrian loss value to obtain the trained detection model.

[0076] The pedestrian loss value reflects the difference between the pedestrian-related features extracted by the initial detection model from the sample pedestrian image and the pedestrian-related features obtained by distinguishing the background features from the sample fusion features; the smaller the pedestrian loss value, the more similar the features extracted by the two are. After the pedestrian loss value is calculated, the parameters of the initial detection model are adjusted according to the pedestrian loss value to obtain the trained detection model. The trained detection model can identify the background and pedestrians from the input image respectively, thereby improving the accuracy of pedestrian detection.

[0077] In some embodiments, Fig. 9 As shown, Fig. 9 yes Figure 8 A flowchart for implementing step 402 in FIG. 4 is provided, wherein the initial detection model is trained according to the background loss value and the pedestrian loss value to obtain the trained detection model, including the following steps 501 to 503: Step 501, fix the second parameters corresponding to the second feature extractor and the pedestrian detection network, and adjust the first parameters corresponding to the first feature extractor and the background detection network according to the background loss value.

[0078] Step 502, fix the first parameters corresponding to the first feature extractor and the background detection network, and adjust the second parameters corresponding to the second feature extractor and the pedestrian detection network according to the pedestrian loss value.

[0079] Step 503, alternately adjusting the first parameter and the second parameter until a preset training condition is reached, thereby obtaining a trained detection model.

[0080] Steps 501 to 503 are described in detail below.

[0081] In some embodiments, Fig.10 As shown, Fig.10This is another optional flow chart of the training method of the detection model provided in an embodiment of the present application, wherein the initial detection model includes a first feature extractor, a second feature extractor, a background detection network and a pedestrian detection network, the initial background features are extracted by the first feature extractor, the initial pedestrian features are extracted by the second feature extractor, the sample background features are predicted by the background detection network, and the sample pedestrian features are predicted by the pedestrian detection network.

[0082] Exemplarily, an optional training method for the initial detection model includes the following steps (1) to (7): (1) Using the first feature extractor to extract the initial background feature A from the sample background image, and using the second feature extractor to extract the initial pedestrian feature a from the sample pedestrian image; (2) Fuse (⊕) the initial background feature A and the initial pedestrian feature a to obtain the sample fusion feature Aa; (3) Based on the background detection network, the sample background feature is extracted from the sample fusion feature Aa, and based on the pedestrian detection network, the sample pedestrian feature is extracted from the sample fusion feature Aa; (4) Calculate the background loss value based on the initial background features and the sample background features, and calculate the pedestrian loss value based on the initial pedestrian features and the sample pedestrian features; (5) fixing the second parameters corresponding to the second feature extractor and the pedestrian detection network, and adjusting the first parameters corresponding to the first feature extractor and the background detection network according to the background loss value, so as to train the background detection network to have the ability to extract background features from the pedestrian and background fusion features; (6) Fixing the first parameters corresponding to the first feature extractor and the background detection network, and adjusting the second parameters corresponding to the second feature extractor and the pedestrian detection network according to the pedestrian loss value, the purpose of which is to train the pedestrian detection network to have the ability to extract pedestrian features from the fusion features of pedestrians and background; (7) Iterate the above steps (1) to (6) until the preset training conditions are met, thereby obtaining a trained detection model.

[0083] It should be noted that if Fig.10 As shown, the first feature extractor and the second feature extractor can also process multiple images in parallel to obtain initial background features B, initial background features N, initial pedestrian features b and initial background features n; and, wherein, the processing steps for the initial background features B, initial background features N, initial pedestrian features b and initial background features n are similar to the above steps (1) to (7), and are not repeated here.

[0084] In some embodiments, Fig.11 As shown, Fig.11 yes Figure 8 Another implementation flowchart of step 402 in the embodiment of the present invention, wherein the initial detection model is trained according to the background loss value and the pedestrian loss value to obtain the trained detection model, further comprising the following steps 601 to 604: Step 601, obtaining the real verification label corresponding to the sample pedestrian image.

[0085] Step 602 , predicting the sample pedestrian image based on the sample pedestrian features to obtain a sample prediction result, and calculating the pedestrian loss value based on the real verification label and the sample prediction result.

[0086] Steps 601 to 602 are described in detail below.

[0087] The real verification label is used to indicate the real pedestrian position and category information in the sample pedestrian image, and is used to evaluate the accuracy of the initial detection model during the training process. The real verification label can be obtained through manual annotation or automatically annotated by a more advanced artificial intelligence model that has been trained.

[0088] Furthermore, after obtaining the sample pedestrian features, the initial detection model will predict the pedestrians in the input sample pedestrian image based on the sample pedestrian features to obtain sample prediction results, and the sample prediction results can be shown by the target box annotation method. It should be noted that in order to facilitate the calculation of the pedestrian loss value, the annotation method of the sample prediction results is usually the same as the real verification label. In addition, the annotation method of pedestrian detection can also be set according to actual conditions, and the embodiment of the present application does not limit this.

[0089] In some embodiments, Fig.12 As shown, Fig.12 yes Fig.11 A flowchart for implementing step 602 in FIG. 1 is provided, wherein the pedestrian loss value is calculated based on the real verification label and the sample prediction result, including the following steps 701 to 704: Step 701, calculate the first square operation result of the difference between the real position information and the predicted position information, and determine the multiplication value of the preset first weight and the first square operation result as the first loss term.

[0090] Step 702, calculate the difference between the square root operation result of the actual size information and the square root operation result of the predicted size information, and determine the multiplication value between the preset second weight and the second square operation result of the result difference as the second loss term.

[0091] Step 703: Calculate the third square operation result of the difference between the predicted category information and the true category information, and determine the multiplication value of the preset third weight and the third square operation result as the third loss term.

[0092] Step 704: Determine the pedestrian loss value as the sum of the first loss item, the second loss item, and the third loss item.

[0093] Steps 701 to 704 are described in detail below.

[0094] In some embodiments, the true verification label includes true position information, true size information, and true category information, and the sample prediction result includes predicted position information, predicted size information, and predicted category information.

[0095] Furthermore, based on the following formula <1> Calculate the pedestrian loss value :

[0096] <1> Among them, i represents the prediction result of the i-th sample and the corresponding true verification label; c represents the preset category information; To predict the position information, it is used to characterize the predicted center position of the sample detection box; It is the predicted size information, which is used to represent the width and height of the sample detection box relative to the predicted center position information; To predict category information, it is used to indicate the probability that the circled target in the sample detection frame is of category c (such as pedestrians); , is the real position information, which is used to represent the real center position of the real detection frame; It is the real size information, which is used to represent the width and height of the real detection box relative to the real center position information; It is the real category information, which is used to indicate the category of the circled target in the real detection frame (such as pedestrian).

[0097] Among them, the true category information and the predicted category information (collectively referred to as "category information") can be adaptively adjusted according to the different image scenes used when the initial detection model is trained. For example, if the image scene is a traffic intersection, the category information can be pedestrians waiting for green lights and pedestrians running red lights.

[0098] in, is the first weight, is the first loss item; is the second weight, is the second loss item; is the third weight, The third loss item.

[0099] It should be noted that the specific values ​​of the first weight, the second weight and the third weight can be adaptively adjusted according to actual conditions, and the embodiments of the present application do not limit this.

[0100] Step 603: Calculate a background loss value based on the initial background features and the sample background features, and determine the difference between the background loss value and the pedestrian loss value as the detection loss value.

[0101] Step 604: train the initial detection model according to the detection loss value to obtain a trained detection model.

[0102] Steps 603 to 604 are described in detail below.

[0103] The background loss value is obtained by comparing the difference between the initial background features and the sample background features, and the background loss value can be calculated by methods such as mean square error function, mean absolute error function, and cross entropy loss function. The specific calculation method of the background loss value is not limited in the embodiment of the present application.

[0104] For example, by the following formula <2> Calculate the background loss value : <2> in, Sample background features predicted by the background detection network; for Norm.

[0105] Furthermore, by the following formula <3> Calculate the detection loss value : L= <3> In an optional model training method, the background loss value is minimized (min) by adjusting the first parameter and the second parameter, so as to train the first feature extractor and the background detection network to have the ability to extract background features, and obtain the corresponding detection loss value in this case; in the next stage of training, the background loss value is maximized (max) by adjusting the first parameter and the second parameter, so as to train the background detection network to extract background features different from the first feature extractor, thereby discovering more features that may be the background, and further helping the first feature extractor to train its feature extraction capability in the next training.

[0106] Furthermore, the trained detection model can filter out redundant background information from the initial pedestrian features based on the initial background features, and can determine the pedestrians that were missed in the sample pedestrian image by comparing the differences between the initial background features and the initial pedestrian features, thereby avoiding misjudgment and missed judgment in pedestrian detection, and thus improving the accuracy of pedestrian detection.

[0107] like Fig.13 As shown, Fig.13801 to 806: Step 801: Obtain a sample pedestrian image and a sample background image that matches the scene in the sample pedestrian image, and input the sample pedestrian image and the sample background image into an initial detection model.

[0108] Step 802, extracting features from the background in the sample background image to obtain initial background features, extracting features from pedestrians in the sample pedestrian image to obtain initial pedestrian features, and fusing the initial background features and initial pedestrian features to obtain sample fusion features.

[0109] Step 803: predict the background in the sample fusion feature to obtain the sample background feature.

[0110] Step 804, a background loss value is calculated based on the sample background features and the initial background features, and an initial detection model is trained according to the background loss value to obtain a trained detection model.

[0111] Step 805 , obtaining a target pedestrian image and a target background image, wherein the target pedestrian image and the sample pedestrian image are different.

[0112] Step 806, input the target pedestrian image and the target background image into the trained detection model to obtain the target detection result.

[0113] Steps 801 to 806 are described in detail below.

[0114] The method of training the detection model in step 801 to step 804 is similar to step 101 to step 104, and will not be repeated here. After obtaining the detection model, the acquired target pedestrian image and target background image are input into the trained detection model, so that the target pedestrian image is processed based on the target background image by the detection model to obtain the target detection result.

[0115] The method for obtaining the target pedestrian image and the method for obtaining the target background image are similar to the method for obtaining the sample pedestrian image and the method for obtaining the sample background image, and are not described here. It should be noted that the target pedestrian image is different from the sample pedestrian image. Specifically, the target pedestrian image does not require the area ratio of the pedestrian and the background in the image.

[0116] Furthermore, the trained first feature extractor is used to extract the starting background features from the target background image, and the trained second feature extractor is used to extract the starting pedestrian features from the target pedestrian image; the starting background features and the starting pedestrian features are fused to obtain the target fused features; then, the target pedestrian features are predicted from the target fused features by the pedestrian detection network; it can be understood that the trained second feature extractor has the ability to extract features that are highly relevant to pedestrians from the target pedestrian image, and, in order to further improve the accuracy of pedestrian detection, the starting background features extracted by the first feature extractor are used to help eliminate redundant background information, so that the pedestrian detection network can better perform pedestrian detection operations on the target pedestrian image based on the target pedestrian features.

[0117] Alternatively, after extracting the initial pedestrian features from the target pedestrian image using the trained second feature extractor, the target pedestrian detection network is directly used to predict the pedestrians in the initial pedestrian features to obtain the target pedestrian features; it can be understood that since the trained second feature extractor and the pedestrian detection network already have the ability to extract pedestrian features from the input information, and the pedestrian feature recognition ability of the pedestrian detection network is stronger than that of the second feature extractor, therefore, based on the second feature extractor and the pedestrian detection network, the detection model can strengthen the feature representation related to pedestrians; based on this, the detection model can predict the target pedestrian image based on the extracted target pedestrian features to obtain the target detection result. Among them, the target detection result includes the predicted pedestrian position and predicted category information corresponding to the input target pedestrian image.

[0118] like Fig.14 As shown, Fig.14 This is an optional functional module diagram of a training device for a detection model provided in an embodiment of the present application, and the device includes: The acquisition module 901 is used to acquire a sample pedestrian image and a sample background image that matches the scene in the sample pedestrian image, and input the sample pedestrian image and the sample background image into an initial detection model.

[0119] The feature extraction module 902 is used to extract features of the background in the sample background image to obtain initial background features, and to extract features of pedestrians in the sample pedestrian image to obtain initial pedestrian features, and to fuse the initial background features and the initial pedestrian features to obtain sample fusion features.

[0120] The prediction module 903 is used to predict the background in the sample fusion feature to obtain the sample background feature.

[0121] The training module 904 is used to calculate the background loss value based on the sample background features and the initial background features, and train the initial detection model according to the background loss value to obtain a trained detection model.

[0122] The present application proposes a detection model training method, a pedestrian detection method, an apparatus and related equipment, which obtain a sample pedestrian image and a sample background image that matches the scene in the sample pedestrian image, and input the sample pedestrian image and the sample background image into an initial detection model; extract features of the background in the sample background image to obtain initial background features, and extract features of the pedestrian in the sample pedestrian image to obtain initial pedestrian features, and fuse the initial background features and the initial pedestrian features to obtain sample fusion features; the initial background features and the initial pedestrian features contain different information, based on which the initial detection model separates the features of the pedestrian and the features of the background, so as to further learn the ability to distinguish between background features and pedestrian features from the sample fusion features mixed with the background and pedestrians after targeted learning of the features of the background and pedestrians respectively; then, predict the background in the sample fusion features to obtain sample background features; calculate a background loss value based on the sample background features and the initial background features, and train the initial detection model according to the background loss value to obtain a trained detection model. The trained detection model can identify the background from the input image, distinguish the background from pedestrians, and perform pedestrian detection operations, thereby improving the accuracy of pedestrian detection.

[0123] The specific implementation of the training device for the detection model is basically the same as the specific implementation of the training method for the above-mentioned detection model, and will not be repeated here.

[0124] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned detection model training method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0125] like Fig.15 As shown, Fig.15 : is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application, the electronic device includes: The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the training method of the detection model of the embodiment of this application; Input / output interface 1003, used to implement information input and output; Communication interface 1004, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.); A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 ); The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0126] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the training method of the above-mentioned detection model is implemented.

[0127] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0128] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0129] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0130] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0131] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0132] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0133] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0135] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0138] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A method for training a detection model, characterized in that: The method comprises: Acquire a sample pedestrian image and a sample background image that matches the scene in the sample pedestrian image, and input the sample pedestrian image and the sample background image into an initial detection model; Extracting features of the background in the sample background image to obtain initial background features, extracting features of pedestrians in the sample pedestrian image to obtain initial pedestrian features, and fusing the initial background features and the initial pedestrian features to obtain sample fusion features; Predicting the background in the sample fusion feature to obtain a sample background feature; A background loss value is calculated based on the sample background features and the initial background features, and the initial detection model is trained according to the background loss value to obtain a trained detection model.

2. The method according to claim 1, characterized in that The step of training the initial detection model according to the background loss value to obtain a trained detection model includes: Predicting pedestrians in the sample fusion features to obtain sample pedestrian features; A pedestrian loss value is calculated based on the sample pedestrian features and the initial pedestrian features, and the initial detection model is trained according to the background loss value and the pedestrian loss value to obtain a trained detection model.

3. The method according to claim 2, characterized in that The initial detection model includes a first feature extractor, a second feature extractor, a background detection network and a pedestrian detection network, the initial background feature is extracted by the first feature extractor, the initial pedestrian feature is extracted by the second feature extractor, the sample background feature is predicted by the background detection network, and the sample pedestrian feature is predicted by the pedestrian detection network; The step of training the initial detection model according to the background loss value and the pedestrian loss value to obtain a trained detection model includes: Fixing second parameters corresponding to the second feature extractor and the pedestrian detection network, and adjusting first parameters corresponding to the first feature extractor and the background detection network according to the background loss value; Fixing first parameters corresponding to the first feature extractor and the background detection network, and adjusting second parameters corresponding to the second feature extractor and the pedestrian detection network according to the pedestrian loss value; The first parameter and the second parameter are adjusted alternately until a preset training condition is reached, thereby obtaining the trained detection model.

4. The method according to claim 3, characterized in that The first feature extractor comprises a plurality of first convolutional layers connected in sequence, each of the first convolutional layers being used to perform a first convolution processing operation on an input image; The extracting features of the background in the sample background image to obtain initial background features includes: Performing at least one first convolution operation on the sample background image, each of the first convolution layers outputs a corresponding first intermediate image, wherein, except for the first first convolution layer, the other first convolution layers perform the first convolution operation on the first intermediate image output by the previous first convolution layer; A feature extraction operation is performed on the first intermediate image output by the last of the first convolutional layers to obtain the initial background features.

5. The method according to claim 3, characterized in that: The second feature extractor comprises a plurality of second convolutional layers connected in sequence, each of the second convolutional layers being used to perform a second convolution processing operation on the input image; The extracting features of pedestrians in the sample pedestrian image to obtain initial pedestrian features includes: Performing at least one second convolution operation on the sample pedestrian image, each of the second convolution layers outputs a corresponding second intermediate image, wherein, except for the first second convolution layer, the other second convolution layers perform the second convolution operation on the second intermediate image output by the previous second convolution layer; A feature extraction operation is performed on the second intermediate image output by the last second convolutional layer to obtain the initial pedestrian features.

6. The method according to claim 2, characterized in that The step of training the initial detection model according to the background loss value and the pedestrian loss value to obtain a trained detection model further includes: Obtaining a true verification label corresponding to the sample pedestrian image; Performing prediction processing on the sample pedestrian image based on the sample pedestrian feature to obtain a sample prediction result, and calculating a pedestrian loss value according to the real verification label and the sample prediction result; Calculating a background loss value according to the initial background features and the sample background features, and determining the difference between the background loss value and the pedestrian loss value as a detection loss value; The initial detection model is trained according to the detection loss value to obtain the trained detection model.

7. The method according to claim 6, characterized in that The real verification label includes real position information, real size information and real category information, and the sample prediction result includes predicted position information, predicted size information and predicted category information; The calculating the pedestrian loss value according to the real verification label and the sample prediction result includes: Calculate a first square operation result of the difference between the actual position information and the predicted position information, and determine a multiplication value of a preset first weight and the first square operation result as a first loss term; Calculate the difference between the square root operation result of the real size information and the square root operation result of the predicted size information, and determine that the multiplication value between the preset second weight and the second square operation result of the result difference is a second loss term; Calculating a third square operation result of the difference between the predicted category information and the true category information, and determining a third loss term as a multiplication value of a preset third weight and the third square operation result; Determine that the sum of the first loss item, the second loss item and the third loss item is the pedestrian loss value.

8. A pedestrian detection method, characterized in that: The method comprises: Acquire a sample pedestrian image and a sample background image that matches the scene in the sample pedestrian image, and input the sample pedestrian image and the sample background image into an initial detection model; Extracting features of the background in the sample background image to obtain initial background features, extracting features of pedestrians in the sample pedestrian image to obtain initial pedestrian features, and fusing the initial background features and the initial pedestrian features to obtain sample fusion features; Predicting the background in the sample fusion feature to obtain a sample background feature; Calculate a background loss value based on the sample background features and the initial background features, and train the initial detection model according to the background loss value to obtain a trained detection model; Acquire a target pedestrian image and a target background image, wherein the target pedestrian image is different from the sample pedestrian image; The target pedestrian image and the target background image are input into the trained detection model to obtain a target detection result.

9. A detection model training device, characterized in that: The device comprises: An acquisition module, used for acquiring a sample pedestrian image and a sample background image matching the scene in the sample pedestrian image, and inputting the sample pedestrian image and the sample background image into an initial detection model; A feature extraction module is used to extract features of the background in the sample background image to obtain initial background features, and to extract features of pedestrians in the sample pedestrian image to obtain initial pedestrian features, and to fuse the initial background features and the initial pedestrian features to obtain sample fusion features; A prediction module, used to predict the background in the sample fusion feature to obtain a sample background feature; A training module is used to calculate a background loss value based on the sample background features and the initial background features, and train the initial detection model according to the background loss value to obtain a trained detection model.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the training method of the detection model described in any one of claims 1 to 7 or the pedestrian detection method described in claim 8 when executing the computer program.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the training method of the detection model described in any one of claims 1 to 7 or the pedestrian detection method described in claim 8.

Images