Target object detection method and device and computer equipment

Abstract

The invention discloses a target object detection method and device and computer equipment. The method comprises the following steps: acquiring a to-be-detected video stream; extracting a to-be-detected image frame from the to-be-detected video stream; a target object in the to-be-detected image frame is detected through a preset model, a detection result of the target object in the to-be-detected image frame is obtained, the detection result is at least used for representing the type of the target object and the position of a detection frame corresponding to the target object, and the preset model at least comprises a classification head and a positioning head; the classification head is used for calculating classification results, and the positioning head is used for determining position coordinates of the detection frame. According to the method and the device, the technical problem of relatively low detection accuracy of a target detection model on a target object in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and more specifically, to a method, apparatus and computer device for detecting a target object. Background Art

[0002] There are many existing technical solutions for video streaming of electric vehicles entering elevators, but the basic technical solutions are roughly as follows: First, a surveillance camera is set up in the elevator, and the target entering the elevator is tracked using target tracking technology. If the tracked target is identified as an electric vehicle in a certain frame, and the probability of it being identified as an electric vehicle is greater than the specified threshold, the alarm system immediately issues an alarm prompt and triggers the elevator brake device, making it impossible for the elevator to close the door normally, in order to remind the passengers to leave the elevator. In the existing technical solutions, the detection algorithm for electric vehicles generally adopts the Yolo (You Only Look Once) algorithm, and the head of the Yolo algorithm is a coupling head, which uses the coupling head to simultaneously complete the classification and positioning of the target (generated in the same set of convolutions). In the process of classification and positioning, the model performance is reduced due to mutual interference, which leads to a low detection accuracy of electric vehicles. Summary of the invention

[0003] The embodiments of the present application provide a target object detection method, apparatus and computer device to at least solve the technical problem that the target detection model in the related art has a low detection accuracy rate for the target object.

[0004] According to one aspect of an embodiment of the present application, a target object detection method is provided, comprising: obtaining a video stream to be detected; extracting an image frame to be detected from the video stream to be detected; using a preset model to detect the target object in the image frame to be detected, and obtaining a detection result of the target object in the image frame to be detected, wherein the detection result is at least used to indicate the type of the target object and the position of a detection frame corresponding to the target object, and the preset model at least includes: a classification head and a positioning head, the classification head is used to calculate the classification result, and the positioning head is used to determine the position coordinates of the detection frame.

[0005] Optionally, a preset model is used to detect the target object in the image frame to be detected, including: using the backbone network in the preset model to perform multiple downsampling on the image frame to be detected with successively increasing downsampling ratios to obtain multiple feature maps with different downsampling ratios corresponding to each image frame; inputting the three feature maps with the largest sampling ratios corresponding to the image frame to be detected into the neck network for processing to obtain output features; inputting the output features into the classification head, the positioning head and the foreground and background head in the preset model for detection to obtain output results, and the output results are used to determine the detection results of the target object, wherein the foreground and background head is used to determine the foreground and background in the image frame to be detected.

[0006] Optionally, the method also includes: comparing the output result with the label data to determine the classification loss value, the positioning loss value and the foreground background loss value; weighted summing the classification loss value, the positioning loss value and the foreground background loss value according to their respective preset weights to obtain a final loss value, wherein the classification loss value is determined by an effective category interval loss function, and the preset weight of the foreground background loss value is the highest.

[0007] Optionally, after outputting the detection result of the target object in the image frame to be detected, the method further includes: marking an identifier for the target object when the target object appears for the first time within a preset time window; creating a state dictionary associated with the identifier, wherein the state dictionary of the target object includes at least: a confidence level that the target object is an object of a target type, and the content in the state dictionary is determined based on the detection result of the target object; and determining whether the target object is an object of the target type at least based on the confidence level that the target object is an object of the target type.

[0008] Optionally, creating a state dictionary associated with the identifier includes: constructing multiple key-value pairs and multiple fields, wherein the multiple key-value pairs include at least: statistical data of multiple types to which the target object belongs, and the multiple fields include at least: a first field, a second field, a third field and a fourth field, the first field is used to represent the current image frame, the second field is used to represent the number of frames in which the target object appears up to the image frame to be detected, the third field is used to represent the moment when the target object first enters the warning area, and the fourth field is used to represent the moment when the target object last enters the warning area, and when a preset alarm condition is triggered, the third field and the fourth field are updated, and the current image frame is updated every preset time period.

[0009] Optionally, the method also includes: extracting the number of times the target object is identified as an object of the target type and the average confidence that the target object is identified as an object of the target type from the statistical data; determining that the target object is an object of the target type when the number of times the target object is identified as an object of the target type is the largest and the average confidence that the target object is identified as an object of the target type is greater than a preset confidence threshold; and determining that the preset alarm condition is triggered when it is detected that the target object is an object of the target type and the target object enters the alarm area.

[0010] Optionally, the warning area is determined in the following manner, including: obtaining an initial warning area, and dividing the initial warning area into multiple rectangular sub-areas; generating an initial matrix based on the positional relationship of the multiple rectangular sub-areas, and setting the value of each element in the initial matrix to zero; using the preset model to perform target detection on each image frame in the video stream to be detected, and when it is determined that the position of the detected target object falls into a target sub-area among the multiple rectangular sub-areas, adding one to the element value corresponding to the target sub-area until the detection time exceeds the preset detection time, so as to obtain a target matrix; and determining the area covered by the rectangular sub-area whose element value in the target matrix exceeds the preset element value threshold as the warning area.

[0011] According to another aspect of an embodiment of the present application, a target object detection device is also provided, including: an acquisition module, used to acquire a video stream to be detected; an extraction module, used to extract image frames to be detected from the video stream to be detected; a detection module, used to detect the target object in the image frame to be detected using a preset model, and obtain a detection result of the target object in the image frame to be detected, wherein the detection result is at least used to indicate the type of the target object and the position of a detection box corresponding to the target object, and the preset model at least includes: a classification head and a positioning head, the classification head is used to calculate the classification result, and the positioning head is used to determine the position coordinates of the detection box.

[0012] According to another aspect of the embodiments of the present application, a computer device is provided, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and is used to execute the above-mentioned target object detection method.

[0013] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned target object detection method by running the computer program.

[0014] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned target object detection method when executed by a processor.

[0015] In an embodiment of the present application, a video stream to be detected is obtained; an image frame to be detected is extracted from the video stream to be detected; a preset model is used to detect the target object in the image frame to be detected, and a detection result of the target object in the image frame to be detected is obtained, wherein the detection result is at least used to indicate the type of the target object and the position of the detection frame corresponding to the target object, and the preset model at least includes: a classification head and a positioning head, the classification head is used to calculate the classification result, and the positioning head is used to determine the position coordinates of the detection frame, so that the head in the preset model is at least split into a classification head and a positioning head, thereby avoiding mutual interference between positioning and classification, thereby achieving the technical effect of improving the accuracy of target detection, and then solving the technical problem of low detection accuracy of target objects by target detection models in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a target object detection method according to an embodiment of the present application;

[0018] Figure 2 is a flow chart of a target object detection method according to an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of a preset model structure according to an embodiment of the present application;

[0020] Figure 4 is a flow chart of another target object detection method according to an embodiment of the present application;

[0021] Figure 5 is a schematic diagram of warning area learning according to an embodiment of the present application;

[0022] Figure 6 A schematic diagram of a detection frame according to an embodiment of the present application;

[0023] Figure 7 A schematic diagram of a target matrix according to an embodiment of the present application;

[0024] Figure 8 It is a structural diagram of a target object detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or reject automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0028] In order to solve the problems existing in the related art, the present application embodiment provides a target object detection method, which can be run on Figure 1 In the computer terminal shown, the computer terminal is explained below.

[0029] The target object detection method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a target object detection method. Figure 1As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0031] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the target object detection method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned target object detection method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0032] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0034] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0035] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a target object detection method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] Figure 2 is a flow chart of a target object detection method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0037] Step S202, obtaining a video stream to be detected;

[0038] In the above step S202, the video stream to be detected may be a video stream in a target area, for example, a video stream in an elevator.

[0039] Step S204, extracting the image frame to be detected from the video stream to be detected;

[0040] Step S206, using a preset model to detect the target object in the image frame to be detected, and obtaining a detection result of the target object in the image frame to be detected, wherein the detection result is at least used to indicate the type of the target object and the position of the detection box corresponding to the target object, and the preset model at least includes: a classification head and a positioning head, the classification head is used to calculate the classification result, and the positioning head is used to determine the position coordinates of the detection box.

[0041] Through the above steps S202 to S206, the video stream to be detected is obtained; the image frame to be detected is extracted from the video stream to be detected; the target object in the image frame to be detected is detected using a preset model to obtain the detection result of the target object in the image frame to be detected, wherein the detection result is at least used to indicate the type of the target object and the position of the detection frame corresponding to the target object, and the preset model at least includes: a classification head and a positioning head, the classification head is used to calculate the classification result, and the positioning head is used to determine the position coordinates of the detection frame, so that the head in the preset model is at least split into a classification head and a positioning head, thereby avoiding the purpose of mutual interference between positioning and classification, thereby achieving the technical effect of improving the accuracy of target detection, and thus solving the technical problem of low detection accuracy of target objects by target detection models in related technologies. The following is a detailed description.

[0042] In step S202 of the target object detection method, a preset model is used to detect the target object in the image frame to be detected, including: using the backbone network in the preset model to perform multiple downsampling of the image frame to be detected with successively increasing downsampling ratios to obtain multiple feature maps with different downsampling ratios corresponding to each image frame; inputting the three feature maps with the largest sampling ratios corresponding to the image frame to be detected into the neck network for processing to obtain output features; inputting the output features into the classification head, the positioning head and the foreground and background head in the preset model for detection to obtain output results, and the output results are used to determine the detection results of the target object, wherein the foreground and background head is used to determine the foreground and background in the image frame to be detected.

[0043] It should be noted that the backbone network is usually composed of multiple convolutional layers, pooling layers and activation functions, which are used to extract features from the input image.

[0044] After obtaining the output result, the output result is compared with the label data to determine the classification loss value, the positioning loss value and the foreground background loss value; the classification loss value, the positioning loss value and the foreground background loss value are weighted and summed according to their respective preset weights to obtain the final loss value, wherein the classification loss value is determined by the effective category interval loss function, and the preset weight of the foreground background loss value is the highest.

[0045] Figure 3 A schematic diagram of the structure of a preset model is shown, such as Figure 3 As shown, the preset model includes a backbone network, a Neck (neck network, an intermediate layer or layer group in the network structure used to connect the feature extraction part and the prediction part), a classification head, a positioning head, and a foreground-background distinction head.

[0046] Taking the downsampling number of 5 times as an example, the backbone network extracts features from the input image (image frame to be detected), and the input image will be downsampled 5 times, corresponding to P1-P5; the features generated by P3-P5 are input to Neck, and Neck will integrate, split and reorganize the features to achieve deep fusion processing of the features; Neck's final output has three groups, each group corresponds to output features with different downsampling ratios; the three groups of output features of Neck are sent to two different data processing streams, one data stream is responsible for calculating the classification results, and the other data stream is responsible for judging whether the target box is the foreground background and calculating the coordinates of the target box. The information of these two data streams is not shared, so the three processes of classification and coordinate calculation, and whether it is the foreground background can be decoupled. Finally, the output results are calculated and compared with the real annotation information, and the classification loss Lcls, coordinate loss Lloc and whether it is the foreground background loss Lobj are calculated respectively. The three losses are summed with certain weights and the model parameters are optimized through back propagation. Among them, Lcls uses ECMLoss (Effective Class-Margins Loss), which is a form of binary cross entropy. After the classification head obtains the output result, the exponential power result of the output is calculated and multiplied by the weight of the actual category corresponding to the output. The final calculation result is sent to the ECMLoss function, and the result obtained is the loss function value of the classification head.

[0047] In some embodiments of the present application, after outputting the detection result of the target object in the image frame to be detected, when the target object appears for the first time within a preset time window, an identifier is marked for the target object; a state dictionary associated with the identifier is created, wherein the state dictionary of the target object includes at least: the confidence that the target object is an object of the target type, and the content in the state dictionary is determined based on the detection result of the target object; and determining whether the target object is an object of the target type at least based on the confidence that the target object is an object of the target type.

[0048] Specifically, the detected target is sent to the tracking algorithm to determine whether the target appears for the first time. If the target appears for the first time, the tracking algorithm will return a new tracking id, referred to as track_id, and the corresponding detected target (target object) is recorded as obj. Since it is the first appearance, an associated state dictionary is constructed for obj. The state dictionary contains the following items:

[0049] 1. The length of the track of the target (how many frames it is tracked for);

[0050] 2. Predict the number of times each category is predicted;

[0051] 3. The prediction is a list of historical detection confidences and average detection confidences for each category;

[0052] 4. The time of the first intrusion into the alarm area (the entire screen is used as the alarm area by default during the cold start phase);

[0053] 5. The time of the most recent intrusion into the alarm area;

[0054] 6. Warning image (image frame containing the target object).

[0055] Among them, creating a state dictionary associated with the identifier includes: constructing multiple key-value pairs and multiple fields, wherein the multiple key-value pairs at least include: statistical data of multiple types to which the target object belongs, and the multiple fields at least include: a first field, a second field, a third field and a fourth field, the first field is used to represent the current image frame, the second field is used to represent the number of frames in which the target object appears up to the image frame to be detected, the third field is used to represent the moment when the target object first enters the warning area, and the fourth field is used to represent the moment when the target object last enters the warning area. When a preset alarm condition is triggered, the third field and the fourth field are updated, and the current image frame is updated every preset time period.

[0056] For example, if the target object types include three categories (for example, people, electric cars, and baby carriages), the state dictionary is as follows:

[0057]

[0058] Among them, "0":{"hits_num":0,"conf_list":[],"mean_conf":0.0} represents a key-value pair of type 0, "1":{"hits_num":0,"conf_list":[],"mean_conf":0.0} represents a key-value pair of type 1, "2":{"hits_num":0,"conf_list":[],"mean_conf":0.0} represents a key-value pair of type 2, hits_num represents the number of times predicted as the corresponding type, conf_list represents the historical detection confidence list, mean_conf represents the average confidence, img_cache represents the warning image frame pushed to the client, which can also be the current image frame (first field), age represents the number of frames in which the target object appears up to the image frame to be detected (second field), first_invade_time represents the third field, and last_invade_time represents the fourth field.

[0059] Among them, the average detection confidence represents the average value of the detection confidence in all frames where obj appears. For example, if obj has appeared in 30 frames, the length of the corresponding conf_list list is 30, and age is also equal to 30. Then, all the values ​​in the conf_list in these 30 frames are added and divided by 30 to obtain the average detection confidence of obj in the corresponding prediction category. The average detection confidence is updated as age increases. first_invade_time represents the time of the first invasion of the alarm area, and last_invade_time represents the time of the last invasion of the alarm area. If obj has been tracked before (first appeared in the historical frame), search the state dictionary of the obj, increment age by 1, increment hits_num of the corresponding prediction category by 1, add the detection confidence of obj in the current frame to the conf_list list, and update the mean_conf field.

[0060] It should be noted that the current image frame is updated once every preset time, and the preset time can be set according to the actual situation, for example, 2 seconds. The converted age is the image frame that passes within 2 seconds. By setting the preset time, the cached image quality is guaranteed and the storage space is saved.

[0061] In another optional method, the length of the time window is set (it is found through manual statistics that the time from most targets entering the elevator to completely closing the elevator door is about 6 seconds, which ensures that more data frames are provided for information fusion and an early warning can be issued before the elevator runs). If it exceeds the time window, the logic of determining whether the alarm area has been invaded is triggered.

[0062] The specific steps of determining whether the target object is an object of the target type at least based on the confidence that the target object is an object of the target type are: extracting the number of times the target object is identified as an object of the target type and the average confidence that the target object is identified as an object of the target type from the statistical data; and determining that the target object is an object of the target type when the number of times the target object is identified as an object of the target type is the largest and the average confidence that the target object is identified as an object of the target type is greater than a preset confidence threshold.

[0063] It should be noted that the target type can be determined according to the actual application scenario, for example: electric vehicles.

[0064] In the case where it is detected that the target object is an object of the target type and the target object enters the warning area, it is determined to trigger the preset warning condition.

[0065] Figure 4 A flow chart of electric vehicle intrusion detection is shown, Figure 4 As shown, the video stream to be detected is detected, and when the time window expires, it is determined whether it is an electric vehicle intrusion.

[0066] In an optional manner, the warning area is determined in the following manner, including: obtaining an initial warning area, and dividing the initial warning area into multiple rectangular sub-areas; generating an initial matrix based on the positional relationship of the multiple rectangular sub-areas, and setting the value of each element in the initial matrix to zero; using the preset model to perform target detection on each image frame in the video stream to be detected, and when it is determined that the position of the detected target object falls into a target sub-area among the multiple rectangular sub-areas, adding one to the element value corresponding to the target sub-area until the detection time exceeds the preset detection time, thereby obtaining a target matrix; and determining the area covered by the rectangular sub-area whose element value in the target matrix exceeds the preset element value threshold as the warning area.

[0067] Figure 5 A schematic diagram of warning area learning is shown in FIG. Figure 5As shown in the figure, the green sector area represents the radiation area of ​​the camera, the gray bar represents the elevator door, the small circles of different colors represent the frequency of the target stopping at the specified position, and the blue rectangular box represents the learned alarm area. After the algorithm is deployed, the captured image of the camera corresponds to a rectangular elevator area of ​​fixed size (the size is generally 1920x1080), and the rectangular area is divided into 10x20 grids vertically and horizontally respectively. A 10x20 matrix M is generated in the memory, and the values ​​of all elements of the matrix are initialized to 0, indicating that no target has stopped; the algorithm continuously detects the target in the image obtained by the camera from the streaming server. If the target is detected, the coordinates of the center points on the left, right and bottom sides of the target box are calculated. These three coordinates are as follows Figure 6 As shown in the figure, the corresponding element value of the corresponding position in M ​​will be incremented by 1, indicating that a target has stopped at this position once. As the algorithm runs, after a period of time (the default is 24 hours, which can be configured), the values ​​of each element in M ​​will be different. The larger the value, the higher the frequency of the target stopping near the corresponding grid, and the greater the probability of belonging to the actual manned area in the elevator. Figure 2 The dark red circles represent the locations with high dwelling frequency in M, the white circles represent the locations with sparse dwelling frequency, and the light red circles represent the locations with moderate dwelling frequency. Based on the prior rule, a threshold is set, the grids above the threshold are selected, and a minimum rectangular box is determined to contain these grids. The final rectangular box can be used as the subsequent warning area, such as Figure 7 As shown, the smallest rectangle defined by grids above 200 is used as the warning area (gray area).

[0068] Figure 8 is a structural diagram of a target object detection device according to an embodiment of the present application, such as Figure 7 As shown, the device comprises:

[0069] An acquisition module 70 is used to acquire a video stream to be detected;

[0070] An extraction module 72, used for extracting the image frame to be detected from the video stream to be detected;

[0071] The detection module 74 is used to detect the target object in the image frame to be detected by using a preset model, and obtain the detection result of the target object in the image frame to be detected, wherein the detection result is at least used to indicate the type of the target object and the position of the detection frame corresponding to the target object, and the preset model at least includes: a classification head and a positioning head, the classification head is used to calculate the classification result, and the positioning head is used to determine the position coordinates of the detection frame.

[0072] Through the above-mentioned target object detection device, by obtaining a video stream to be detected; extracting an image frame to be detected from the video stream to be detected; using a preset model to detect the target object in the image frame to be detected, and obtaining a detection result of the target object in the image frame to be detected, wherein the detection result is at least used to indicate the type of the target object and the position of the detection frame corresponding to the target object, and the preset model at least includes: a classification head and a positioning head, the classification head is used to calculate the classification result, and the positioning head is used to determine the position coordinates of the detection frame, so that the head in the preset model is at least split into a classification head and a positioning head, thereby avoiding the purpose of mutual interference between positioning and classification, thereby achieving the technical effect of improving the accuracy of target detection, and then solving the technical problem of low detection accuracy of target objects by target detection models in related technologies.

[0073] The detection module 74 in the above-mentioned target object detection device includes: a detection submodule, which is used to detect the target object in the image frame to be detected using a preset model, including: using the backbone network in the preset model to perform multiple downsampling of the image frame to be detected with successively increasing downsampling ratios to obtain multiple feature maps with different downsampling ratios corresponding to each image frame; inputting the three feature maps with the largest sampling ratios corresponding to the image frame to be detected into the neck network for processing to obtain output features; inputting the output features into the classification head, the positioning head and the foreground and background head in the preset model for detection to obtain output results, and the output results are used to determine the detection results of the target object, wherein the foreground and background head is used to determine the foreground and background in the image frame to be detected.

[0074] The detection submodule includes: a loss submodule, which is used to compare the output result with the label data to determine the classification loss value, the positioning loss value and the foreground background loss value; the classification loss value, the positioning loss value and the foreground background loss value are weighted and summed according to their respective preset weights to obtain a final loss value, wherein the classification loss value is determined by an effective category interval loss function, and the preset weight of the foreground background loss value is the highest.

[0075] The detection module 74 includes: a state dictionary submodule, which is used to mark an identifier for the target object when the target object appears for the first time within a preset time window after outputting the detection result of the target object in the image frame to be detected; create a state dictionary associated with the identifier, wherein the state dictionary of the target object includes at least: the confidence that the target object is an object of the target type, and the content in the state dictionary is determined based on the detection result of the target object; and determine whether the target object is an object of the target type at least based on the confidence that the target object is an object of the target type.

[0076] The status dictionary submodule includes: a creation unit, which is used to create a status dictionary associated with the identifier, including: constructing multiple key-value pairs and multiple fields, wherein the multiple key-value pairs at least include: statistical data of multiple types to which the target object belongs, and the multiple fields at least include: a first field, a second field, a third field and a fourth field, the first field is used to represent the current image frame, the second field is used to represent the number of frames in which the target object appears up to the image frame to be detected, the third field is used to represent the moment when the target object first enters the warning area, and the fourth field is used to represent the moment when the target object last enters the warning area. When a preset alarm condition is triggered, the third field and the fourth field are updated, and the current image frame is updated every preset time period.

[0077] A creation unit, including an alarm subunit and a determination subunit, for extracting from the statistical data the number of times the target object is identified as an object of the target type and an average confidence level that the target object is identified as an object of the target type; determining that the target object is an object of the target type when the number of times the target object is identified as an object of the target type is the largest and the average confidence level that the target object is identified as an object of the target type is greater than a preset confidence threshold; and determining that the preset alarm condition is triggered when it is detected that the target object is an object of the target type and the target object enters the alarm area.

[0078] A determination subunit is provided, which is used to obtain an initial warning area and divide the initial warning area into a plurality of rectangular sub-areas; an initial matrix is ​​generated according to the positional relationship of the plurality of rectangular sub-areas, and the value of each element in the initial matrix is ​​set to zero; the preset model is used to perform target detection on each image frame in the video stream to be detected, and when it is determined that the position of the detected target object falls into a target sub-area among the plurality of rectangular sub-areas, the element value corresponding to the target sub-area is increased by one until the detection duration exceeds the preset detection duration, thereby obtaining a target matrix; the area covered by the rectangular sub-area whose element value in the target matrix exceeds the preset element value threshold is determined as the warning area.

[0079] It should be noted that Figure 8 The target object detection device shown is used to perform Figure 2 The target object detection method shown in the figure, therefore the relevant explanations in the above target object detection method are also applicable to the target object detection device, and will not be repeated here.

[0080] An embodiment of the present application further provides a computer device, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and is used to execute the above-mentioned target object detection method.

[0081] An embodiment of the present application further provides a non-volatile storage medium, which includes a stored computer program, wherein a device where the non-volatile storage medium is located executes the above-mentioned target object detection method by running the computer program.

[0082] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the target object detection method in the present application are implemented.

[0083] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the target object detection method in the present application.

[0084] The embodiment of the present application also provides a computer program, which, when executed by a processor, implements the steps of the target object detection method in the present application.

[0085] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0086] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0089] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0090] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly or all or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0091] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A target object detection method, characterized in that: include: Get the video stream to be detected; Extracting the image frame to be detected from the video stream to be detected; A preset model is used to detect the target object in the image frame to be detected, and a detection result of the target object in the image frame to be detected is obtained, wherein the detection result is at least used to indicate the type of the target object and the position of the detection frame corresponding to the target object, and the preset model at least includes: a classification head and a positioning head, the classification head is used to calculate the classification result, and the positioning head is used to determine the position coordinates of the detection frame.

2. The method according to claim 1, characterized in that Using a preset model to detect the target object in the image frame to be detected includes: The backbone network in the preset model is used to perform multiple downsampling of the image frame to be detected with successively increasing downsampling ratios, so as to obtain multiple feature maps with different downsampling ratios corresponding to each image frame; Inputting three feature maps with the largest sampling ratio corresponding to the image frame to be detected into the neck network for processing to obtain output features; The output features are respectively input into the classification head, the positioning head and the foreground and background head in the preset model for detection to obtain output results, and the output results are used to determine the detection results of the target object, wherein the foreground and background head is used to determine the foreground and background in the image frame to be detected.

3. The method according to claim 2, characterized in that The method further comprises: Compare the output result with the label data to determine the classification loss value, the positioning loss value and the foreground background loss value; The classification loss value, the positioning loss value and the foreground background loss value are weighted and summed according to their respective preset weights to obtain a final loss value, wherein the classification loss value is determined by an effective category interval loss function, and the preset weight of the foreground background loss value is the highest.

4. The method according to claim 1, characterized in that: After outputting the detection result of the target object in the image frame to be detected, the method further includes: When the target object appears for the first time within a preset time window, marking an identifier for the target object; Creating a state dictionary associated with the identifier, wherein the state dictionary of the target object includes at least: a confidence level that the target object is an object of a target type, and the content of the state dictionary is determined according to a detection result of the target object; Whether the target object is an object of the target type is determined at least according to a confidence level that the target object is an object of the target type.

5. The method according to claim 4, characterized in that Create a state dictionary associated with the identifier, including: Construct multiple key-value pairs and multiple fields, wherein the multiple key-value pairs at least include: statistical data of multiple types to which the target object belongs, and the multiple fields at least include: a first field, a second field, a third field and a fourth field, the first field is used to represent the current image frame, the second field is used to represent the number of frames in which the target object appears up to the image frame to be detected, the third field is used to represent the moment when the target object first enters the warning area, and the fourth field is used to represent the moment when the target object last enters the warning area. When a preset alarm condition is triggered, the third field and the fourth field are updated, and the current image frame is updated every preset time period.

6. The method according to claim 5, characterized in that The method further comprises: Extracting from the statistical data the number of times the target object is identified as an object of the target type and the average confidence level that the target object is identified as an object of the target type; In a case where the target object is identified as an object of the target type the most times and the average confidence level of the target object being identified as an object of the target type is greater than a preset confidence threshold, determining that the target object is an object of the target type; In the case where it is detected that the target object is an object of the target type and the target object enters the warning area, it is determined to trigger the preset warning condition.

7. The method according to claim 6, characterized in that The warning area is determined by: Acquire an initial warning area, and divide the initial warning area into a plurality of rectangular sub-areas; Generate an initial matrix according to the positional relationship of the plurality of rectangular sub-regions, wherein the value of each element in the initial matrix is ​​set to zero; Using the preset model to perform target detection on each image frame in the video stream to be detected, when determining that the position of the detected target object falls into a target sub-region among the multiple rectangular sub-regions, adding one to the element value corresponding to the target sub-region until the detection duration exceeds the preset detection duration, thereby obtaining a target matrix; An area covered by a rectangular sub-area whose element values ​​in the target matrix exceed a preset element value threshold is determined as the warning area.

8. A target object detection device, characterized in that: include: An acquisition module, used to acquire the video stream to be detected; An extraction module, used for extracting the image frame to be detected from the video stream to be detected; A detection module is used to detect the target object in the image frame to be detected using a preset model, and obtain a detection result of the target object in the image frame to be detected, wherein the detection result is at least used to indicate the type of the target object and the position of the detection frame corresponding to the target object, and the preset model at least includes: a classification head and a positioning head, the classification head is used to calculate the classification result, and the positioning head is used to determine the position coordinates of the detection frame.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; The processor is connected to the memory, and is used to execute the target object detection method described in any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the target object detection method according to any one of claims 1 to 7 is implemented.

Images