Method and device for determining target object, storage medium and electronic device

Abstract

The present invention discloses a method and device for determining a target object, a storage medium and an electronic device, wherein the method comprises: performing semantic segmentation processing on first image data through a semantic segmentation model to obtain multiple target areas of the first image data, and determining a first area from the multiple target areas; performing segmentation processing on the first area in the first image data through an instance segmentation model to obtain a first target object in the first area, and type information of the first target object; matching the type information of the first target object with preset target object type information, and determining that the first target object is a target object when the type of the first target object matches the preset target object type. The above technical solution solves the problem that the target object cannot be accurately determined in a segmentation algorithm that only distinguishes between two categories, target object and background, in a traditional detection algorithm.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to a method and device for determining a target object, a storage medium and an electronic device. Background Art

[0002] With the rapid development of society, there are more and more vehicles on the road. Due to some reasons, different types of target objects will appear on the road. In order to avoid the target objects affecting the road traffic, it is necessary to identify the target objects on the road.

[0003] In the traditional target object recognition algorithm, patent CN112330658A performs a binary classification on each pixel to determine whether it is a target object, and uses two high and low thresholds to segment the position of the target object. After removing areas such as lane lines, the real target object area is generated, and finally a classification model is used to further confirm whether it is a real target object; patent CN11205338A uses a semantic segmentation model to obtain road information, and then uses background modeling and module update strategies to remove the area of ​​moving objects. If there is a non-moving area with a large difference between the foreground and background, it is determined to be a target object area; patent CN112149649B uses a detection model to obtain the area of ​​the vehicle, and compares the pictures of the area of ​​the same vehicle in multiple time periods. If the similarity of the difference range before and after the same vehicle is greater than a threshold, it is determined that there is a target object in the area; patent CN111523536A uses a faster rcnn model to detect target objects.

[0004] In the above-mentioned prior art, it can be summarized into three categories of methods, among which the first category gives background modeling and foreground extraction, considers the target object as the foreground, and then filters according to various filtering conditions. However, this method relies on background modeling and will fail for scenes where the camera moves; the second category is a detection-based solution, which uses various detectors to detect the target object, and then classifies and filters the wrong samples according to various conditions. However, due to the different shapes of the target object remains, there are errors in describing the target object using the traditional detection frame, and the effect is not good. The third category is a segmentation solution, which determines whether it is a target object pixel by pixel, and then uses the classification model for secondary filtering to retain the real target object and reduce false detection. Although this method solves the problem of morphology, it simply and crudely divides the two categories of target objects and non-target objects and then classifies them. Due to the large number of target object categories, the direct description of one category will not be very good, and the interpretability is also poor. In addition, the segmentation algorithm uses classification as the final filtering method. This method is also relatively simple and crude, and the interpretability is also relatively poor, and the essential characteristics of the target object are not captured.

[0005] In addition, although the above methods can all perform target object detection, the above methods cannot accurately determine whether the first area has a target object.

[0006] Regarding related technologies, the traditional detection algorithm only distinguishes between two categories, the target object and the background, in the segmentation algorithm, and cannot accurately determine the target object. No effective solution has been proposed yet.

[0007] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the invention

[0008] The embodiments of the present invention provide a method and device for determining a target object, a storage medium and an electronic device, so as to at least solve the problem in the related art that the target object cannot be accurately determined in the segmentation algorithm that only distinguishes between the target object and the background in the traditional detection algorithm.

[0009] According to an embodiment of the present invention, a method for determining a target object is provided, comprising: performing semantic segmentation processing on first image data through a semantic segmentation model to obtain multiple target regions of the first image data, and determining a first region from the multiple target regions; performing instance segmentation processing on the first region in the first image data through an instance segmentation model to obtain a first target object in the first region, and type information of the first target object; matching the type information of the first target object with preset target object type information, and determining that the first target object is a target object when the type information of the first target object matches the preset target object type information.

[0010] In an exemplary embodiment, after determining that the first target object is a target object, the method further includes: loopingly determining whether a second target object consistent with the type information of the first target object exists at the second position of any image data in a video stream of a preset duration, until determining whether a second target object consistent with the type information of the first target object exists at the second position of all image data in the video stream of the preset duration, wherein any image data and the second image data are image data in the same video stream, and a distance between the second position and the first position is less than the preset threshold; when a second target object consistent with the type information of the first target object exists at the second position of all image data, the first target object is determined to be a target object.

[0011] In an exemplary embodiment, looping to determine whether a second target object consistent with the type information of the first target object exists at the second position of any image data in a video stream of a preset duration includes at least one of the following: acquiring second image data, wherein the second image data and the first image data are image data of a preset number of adjacent frames in the same video stream; determining a second target object in a second target area of ​​the second image data, and basic information of the second target object, wherein the basic information of the second target object includes: type information of the second target object and a second position of the second target object, and the area type of the second target area is the same as the area type of the first area; determining whether the type information of the second target object is consistent with the type information of the first target object, and whether the second position of the second target object is less than a preset threshold distance from the first position of the first target object.

[0012] In an exemplary embodiment, after determining whether the type information of the second target object is consistent with the type information of the first target object, and whether the second position of the second target object is less than a preset threshold value from the first position of the first target object, the method further includes: when the type information of the second target object is consistent with the type information of the first target object, and the distance between the second position of the second target object and the first position of the first target object is less than a preset threshold value, determining that there is a second target object consistent with the type information of the first target object at the second position of the second image data.

[0013] In an exemplary embodiment, semantic segmentation is performed on first image data through a semantic segmentation model to obtain multiple target regions of the first image data, including: determining global information of the first image data and local information of the first image data through the semantic segmentation model; segmenting the multiple target regions in the first image data according to the local information, and determining region types of the multiple target regions according to the global information.

[0014] In an exemplary embodiment, a first region is determined from the multiple target regions, and the method further includes: determining a region type that matches a preset region type among the region types of the multiple target regions; and performing dilation and erosion operations on the target region corresponding to the region type that matches the preset region type by a morphological method to obtain the first region.

[0015] In an exemplary embodiment, after matching the type of the first target object with a preset target object type, the method further includes: if the type of the first target object does not match the preset target object type, sending the type of the first target object to a target terminal so that the target object determines again whether the first target object is a target object.

[0016] According to another embodiment of the embodiments of the present invention, there is also provided a device for determining a target object, comprising: a first acquisition module, used to perform semantic segmentation processing on first image data through a semantic segmentation model to obtain multiple target areas of the first image data, and determine a first area from the multiple target areas; a second acquisition module, used to perform instance segmentation processing on the first area in the first image data through an instance segmentation model to obtain a first target object in the first area, and type information of the first target object; a determination module, used to match the type information of the first target object with preset target object type information, and determine that the first target object is a target object when the type information of the first target object matches the preset target object type information.

[0017] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining the target object when running.

[0018] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the target object determination method through the computer program.

[0019] In an embodiment of the present invention, semantic segmentation is performed on first image data through a semantic segmentation model to obtain multiple target areas of the first image data, and a first area is determined from the multiple target areas; instance segmentation is performed on the first area in the first image data through an instance segmentation model to obtain a first target object in the first area, and type information of the first target object; the type information of the first target object is matched with preset target object type information, and when the type information of the first target object matches the preset target object type information, the first target object is determined to be a target object; the above-mentioned technical solution is adopted to solve the problem that the target object cannot be accurately determined in the segmentation algorithm that only distinguishes between the target object and the background in the traditional detection algorithm, and the type information of the target area and the type information of the target object are determined by using a two-level segmentation model, so as to accurately judge whether the target area in the first image data has a target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 It is a hardware structure block diagram of a computer terminal of a method for determining a target object according to an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a method for determining a target object according to an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of a method for determining a target object according to an embodiment of the present invention;

[0024] Figure 4 It is a structural block diagram of a device for determining a target object according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a computer terminal of a method for determining a target object according to an embodiment of the present invention. Figure 1As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0028] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the target object in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] In this embodiment, a method for determining a target object is provided, which is applied to the above-mentioned computer terminal. Figure 2 is a flow chart of a method for determining a target object according to an embodiment of the present invention, and the flow chart includes the following steps:

[0031] Step S202, performing semantic segmentation processing on the first image data by using a semantic segmentation model to obtain multiple target regions of the first image data, and determining a first region from the multiple target regions;

[0032] It should be noted that the above-mentioned target areas can be: road surface areas, green belt areas, isolation fence areas and other areas. The first area is determined from multiple target areas according to the type of the target area, wherein the first area is the road surface area. In the actual practical process, in order to make the performance real-time and effective, a two-way model similar to BiSenet can be used for semantic segmentation.

[0033] Step S204, segmenting the first region in the first image data by using an instance segmentation model to obtain a first target object in the first region and type information of the first target object;

[0034] It should be noted that the type information of the first target object may be a box, a plastic bag, a tire brick, etc., which is not limited in the embodiment of the present invention.

[0035] Step S206: Match the type information of the first target object with preset target object type information. If the type information of the first target object matches the preset target object type information, determine that the first target object is a target object.

[0036] It should be noted that the target object is the scattered objects.

[0037] Through the above steps, semantic segmentation is performed on the first image data through a semantic segmentation model to obtain multiple target areas of the first image data, and a first area is determined from the multiple target areas; instance segmentation is performed on the first area in the first image data through an instance segmentation model to obtain a first target object in the first area, and type information of the first target object; the type information of the first target object is matched with preset target object type information, and when the type information of the first target object matches the preset target object type information, the first target object is determined to be a target object, which solves the problem in the related art that the target object cannot be accurately determined in the segmentation algorithm that only distinguishes between the target object and the background in the traditional detection algorithm, and then the type information of the target area and the type information of the target object are determined by using a two-level segmentation model, so that it can be accurately judged whether the target area in the first image data has a target object.

[0038] It should be noted that semantic segmentation is the classification of image data at the pixel level. Pixels belonging to the same category are classified into one category. For example, the road surface in the first image data belongs to one category, and the green belt belongs to another category. Instance segmentation is to distinguish different instances in the image data. Different instances of the same category are marked with different labels. For example, when there are two plastic bags in the first image data, the two plastic bags have different labels.

[0039] In an exemplary embodiment, after the above step S208, it is cyclically determined whether a second target object consistent with the type information of the first target object exists at the second position of any image data in a video stream of a preset duration, until it is determined whether a second target object consistent with the type information of the first target object exists at the second position of all image data in the video stream of the preset duration, wherein any image data and the second image data are image data in the same video stream, and a distance between the second position and the first position is less than the preset threshold; when a second target object consistent with the type information of the first target object exists at the second position of all image data, the first target object is determined to be a target object.

[0040] That is to say, the video stream captured by the camera is obtained, and the image data of each frame in the video stream or multiple image data at intervals of a preset number of frames are obtained, and multiple image data of the video stream are obtained within the preset number of frames. In this way, frequent operations can be reduced, but it can also be determined whether the target object has always existed in the first area. The preset number of frames can be in units of 10. That is, an image data of the video stream is obtained every 10 frames, and then it is determined whether there is a second target object of the same type as the first target object at the second position of the first area in the image data, and there is a second target object of the same type as the first target object at the second position of the first area in the image data, and the first target object is determined to be the target object. If the accuracy of determining the target object is to be improved as much as possible, the image data of each frame in the video stream can be obtained, and semantic segmentation and instance segmentation are performed on each frame of image data.

[0041] In an exemplary embodiment, looping to determine whether a second target object consistent with the type information of the first target object exists at the second position of any image data in a video stream of a preset duration includes at least one of the following: acquiring second image data, wherein the second image data and the first image data are image data of a preset number of adjacent frames in the same video stream; determining a second target object in a second target area of ​​the second image data, and basic information of the second target object, wherein the basic information of the second target object includes: type information of the second target object and a second position of the second target object, and the area type information of the second target area is the same as the area type information of the first area; determining whether the type information of the second target object is consistent with the type information of the first target object, and whether the second position of the second target object is less than a preset threshold distance from the first position of the first target object.

[0042] That is, the second image data is segmented by a semantic segmentation model to obtain multiple second target areas of the second image data, and a first area is determined from the multiple second target areas; the first area in the second image data is segmented by an instance segmentation model to obtain a second target object in the first area, and the type of the second target object; the type of the second target object is matched with a preset target object type, and when the type of the second target object matches the preset target object type, it is determined whether the type of the second target object is consistent with the type of the first target object, and whether the distance between the second position of the second target object in the second image data and the first position of the first target object in the first image data is less than a preset threshold.

[0043] Specifically, when the type information of the second target object is consistent with the type information of the first target object, and the distance between the second position of the second target object and the first position of the first target object is less than a preset threshold, it is determined that there is a second target object consistent with the type information of the first target object at the second position of the second image data.

[0044] That is to say, when the type information of the second target object is consistent with the type information of the first target object, and the distance between the second position of the second target object and the first position of the first target object is less than a preset threshold, it is determined that the suspected target object at the same position has always existed for a period of time, then it is determined that the suspected target object at the same position has always existed, and the suspected target object is determined as the target object, completing some false detection filtering; if the type information of the second target object is inconsistent with the type information of the first target object, and / or the distance between the second position of the second target object and the first position of the first target object is greater than a preset threshold, it is determined that there is no target object in the first area. By detecting the image data of the preset time period, the accuracy of determining the target object can be improved and the possibility of false detection can be reduced.

[0045] In an exemplary embodiment, semantic segmentation is performed on first image data through a semantic segmentation model to obtain multiple target regions of the first image data, including: determining global information of the first image data and local information of the first image data through the semantic segmentation model; segmenting the multiple target regions in the first image data according to the local information, and determining region types of the multiple target regions according to the global information.

[0046] Specifically, the first image is segmented in the following manner: global information of the first image data is determined through one path of a semantic segmentation model, wherein the global information is used to determine types of multiple target areas in the first image data; local information of the first image data is determined through another path of a semantic model, wherein the local information is used to determine boundaries of multiple target areas, so as to segment the first image data.

[0047] In an exemplary embodiment, a first region is determined from the multiple target regions, and a region type matching a preset region type is determined among the region types of the multiple target regions; and dilation and erosion operations are performed on the target region corresponding to the region type matching the preset region type through a morphological method to obtain the first region.

[0048] The target area is expanded and eroded to eliminate noise in the first image data, calculate the gradient of the image, and then segment out an independent first area. Specifically, the expansion and erosion are to convolve a part of the image or graphic with a kernel. The kernel can be of any shape and size, and it is a separately defined reference point.

[0049] In an exemplary embodiment, after matching the type information of the first target object with the preset target object type information, if the type information of the first target object does not match the preset target object type information, the type information of the first target object is sent to the target terminal so that the target object can determine again whether the first target object is the target object.

[0050] That is, when it is determined that the first target object in the first image is not the target object, the type of the first target object is sent to the terminal device so that the monitoring personnel can confirm whether it is the target object, thereby further improving the accuracy of determining the target object.

[0051] In order to better understand the process of the above-mentioned target object determination method, the implementation method flow of the above-mentioned target object determination is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present invention.

[0052] In this embodiment, a method for determining a target object is provided. Figure 3 is a schematic diagram of a method for determining a target object according to an embodiment of the present invention. Figure 3 As shown, the specific steps are as follows:

[0053] Step S301: accessing the monitored real-time video stream;

[0054] Step S302: obtaining a picture from the video stream (equivalent to the first image data in the above embodiment);

[0055] Step S303: using the pre-trained semantic segmentation model to obtain semantic segmentation results, the types of results include road surface, green belt, isolation fence and other area types;

[0056] The semantic segmentation model is a deep convolutional neural network model. In order to achieve real-time performance, a dual-path model similar to BiSenet is used. The global and local dual-path fusion is used to maximize the accuracy of the model's regional segmentation of the road surface, green belt, and isolation fence.

[0057] Step S304: using morphological methods, dilation, erosion and connected domain analysis to obtain a segmentation mask of the entire road surface area, so as to segment the road surface area through the segmentation mask;

[0058] Step S305: using the pre-trained instance segmentation model, perform instance segmentation on the road surface (equivalent to the first area in the above embodiment), and the instance types to be segmented include: boxes, plastic bags, tire bricks, etc. (equivalent to the first target object in the above embodiment);

[0059] Step S306: selecting which instance types of objects are determined as spilled objects according to the configuration, and recording the results of the spilled objects as suspected spilled objects in the cache;

[0060] Step S307: Based on the recognition results of the image data in the video stream within a period of time T, if it is found that the suspected spilled objects are always present at the same position, it is determined that there are stationary spilled objects, and the suspected spilled objects are determined as spilled objects (equivalent to the target object in the above embodiment), completing the filtering of some false detections.

[0061] Step S308: End.

[0062] An optional embodiment of the present invention utilizes a segmentation model to solve the problem of inaccurate detection frames of irregular spilled objects by the detection model; utilizes a two-level segmentation model to solve the problem of inaccurate determination of spilled objects in a segmentation algorithm that only distinguishes between spilled objects and background categories; utilizes instance segmentation instead of semantic segmentation to detect suspected spilled objects, thereby improving the segmentation and distinction performance under multiple spilled objects; refines the spilled object segmentation categories, and through flexible configuration, can flexibly achieve a balance between the sensitivity of spilled object detection and false detections; utilizes the residence time, a time dimension information of the spilled objects, to replace the original classification and filtering model, thereby better improving the ability to eliminate false detections.

[0063] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0064] In the present embodiment, a device for determining a target object is also provided, and the device for determining a target object is used to realize the above-mentioned embodiment and the preferred embodiment, and the description thereof will not be repeated. As used below, the term "module" can realize a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably realized with software, the realization of hardware, or a combination of software and hardware is also possible and contemplated.

[0065] Figure 4 is a structural block diagram of a device for determining a target object according to an embodiment of the present invention; Figure 4 As shown, including:

[0066] A first acquisition module 42 is used to perform semantic segmentation processing on the first image data through a semantic segmentation model to obtain multiple target regions of the first image data, and determine a first region from the multiple target regions;

[0067] A second acquisition module 44 is used to perform instance segmentation processing on the first region in the first image data by using an instance segmentation model to obtain a first target object in the first region and type information of the first target object;

[0068] The determination module 46 is configured to match the type information of the first target object with preset target object type information, and determine that the first target object is a target object when the type information of the first target object matches the preset target object type information.

[0069] Through the above-mentioned device, semantic segmentation processing is performed on the first image data through a semantic segmentation model to obtain multiple target areas of the first image data, and a first area is determined from the multiple target areas; instance segmentation processing is performed on the first area in the first image data through an instance segmentation model to obtain a first target object in the first area, and type information of the first target object; the type information of the first target object is matched with preset target object type information, and when the type information of the first target object matches the preset target object type information, the first target object is determined to be a target object, which solves the problem in the related art that the target object cannot be accurately determined in the segmentation algorithm that only distinguishes between the target object and the background in the traditional detection algorithm, and then the type information of the target area and the type information of the target object are determined by using a two-level segmentation model, so that it can be accurately judged whether the target area in the first image data has a target object.

[0070] In an exemplary embodiment, the determination module 46 is also used to cyclically determine whether a second target object consistent with the type information of the first target object exists at the second position of any image data in a video stream of a preset duration, until it is determined whether a second target object consistent with the type information of the first target object exists at the second position of all image data in the video stream of the preset duration, wherein any image data and the second image data are image data in the same video stream, and the distance between the second position and the first position is less than the preset threshold; when a second target object consistent with the type information of the first target object exists at the second position of all image data, the first target object is determined to be a target object.

[0071] In an exemplary embodiment, the determination module 46 is also used to obtain second image data, wherein the second image data and the first image data are image data of a preset number of adjacent frames in the same video stream; determine a second target object in a second target area of ​​the second image data, and basic information of the second target object, wherein the basic information of the second target object includes: type information of the second target object and a second position of the second target object, and the area type of the second target area is the same as the area type of the first area; determine whether the type information of the second target object is consistent with the type information of the first target object, and whether the second position of the second target object is less than a preset threshold value from the first position of the first target object.

[0072] In an exemplary embodiment, the determination module 46 is also used to determine that there is a second target object consistent with the type information of the first target object at the second position of the second image data when the type information of the second target object is consistent with the type information of the first target object and the distance between the second position of the second target object and the first position of the first target object is less than a preset threshold.

[0073] In an exemplary embodiment, the first acquisition module 42 is also used to determine the global information of the first image data and the local information of the first image data through the semantic segmentation model; segment the multiple target areas in the first image data according to the local information, and determine the area types of the multiple target areas according to the global information.

[0074] In an exemplary embodiment, the first acquisition module 42 is also used to determine a region type that matches a preset region type among the region types of the multiple target regions; and to perform dilation and erosion operations on the target region corresponding to the region type that matches the preset region type through a morphological method to obtain the first region.

[0075] In an exemplary embodiment, the determination module 46 is also used to send the type information of the first target object to the target terminal when the type information of the first target object does not match the preset target object type information, so that the target object can re-determine whether the first target object is the target object.

[0076] An embodiment of the present invention further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0077] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0078] S1, performing semantic segmentation processing on first image data through a semantic segmentation model to obtain multiple target regions of the first image data, and determining a first region from the multiple target regions;

[0079] S2, performing instance segmentation processing on a first region in the first image data by using an instance segmentation model to obtain a first target object in the first region and type information of the first target object;

[0080] S3: Match the type information of the first target object with preset target object type information, and if the type information of the first target object matches the preset target object type information, determine that the first target object is a target object.

[0081] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0082] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0083] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0084] S1, performing semantic segmentation processing on first image data through a semantic segmentation model to obtain multiple target regions of the first image data, and determining a first region from the multiple target regions;

[0085] S2, performing instance segmentation processing on a first region in the first image data by using an instance segmentation model to obtain a first target object in the first region and type information of the first target object;

[0086] S3: Match the type information of the first target object with preset target object type information, and if the type information of the first target object matches the preset target object type information, determine that the first target object is a target object.

[0087] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0088] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0089] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for determining a target object, characterized in that: include: Performing semantic segmentation processing on the first image data by using a semantic segmentation model to obtain multiple target regions of the first image data, and determining a first region from the multiple target regions; Performing instance segmentation processing on a first region in the first image data by using an instance segmentation model to obtain a first target object in the first region and type information of the first target object; matching the type information of the first target object with preset target object type information, and determining that the first target object is a target object when the type information of the first target object matches the preset target object type information; Among them, semantic segmentation processing is performed on the first image data through a semantic segmentation model to obtain multiple target areas of the first image data, including: determining the global information of the first image data and the local information of the first image data through the semantic segmentation model; segmenting the multiple target areas in the first image data according to the local information, and determining the area types of the multiple target areas according to the global information, wherein the semantic segmentation model is a two-way semantic segmentation model.

2. The method for determining a target object according to claim 1, characterized in that: After determining that the first target object is a target subject, the method further includes: cyclically determining whether a second target object consistent with the type information of the first target object exists at the second position of any image data in a video stream of a preset duration, until determining whether a second target object consistent with the type information of the first target object exists at the second position of all image data in the video stream of the preset duration, wherein any image data and the second image data are image data in the same video stream, and a distance between the second position and the first position is less than a preset threshold; In a case where a second target object having type information consistent with that of the first target object exists at the second position of all the image data, the first target object is determined to be a target object.

3. The method for determining a target object according to claim 2, characterized in that: The step of cyclically determining whether a second target object consistent with the type information of the first target object exists at a second position of any image data in a video stream of a preset duration includes at least one of the following: Acquire second image data, wherein the second image data and the first image data are image data of adjacent preset frames in the same video stream; Determine a second target object in a second target area of ​​the second image data, and basic information of the second target object, wherein the basic information of the second target object includes: type information of the second target object and a second position of the second target object, and the area type of the second target area is the same as the area type of the first area; It is determined whether the type information of the second target object is consistent with the type information of the first target object, and whether the distance between the second position of the second target object and the first position of the first target object is less than a preset threshold.

4. The method for determining a target object according to claim 3, characterized in that: After determining whether the type information of the second target object is consistent with the type information of the first target object, and whether the distance between the second position of the second target object and the first position of the first target object is less than a preset threshold, the method further includes: When the type information of the second target object is consistent with the type information of the first target object, and the distance between the second position of the second target object and the first position of the first target object is less than a preset threshold, it is determined that there is a second target object consistent with the type information of the first target object at the second position of the second image data.

5. The method for determining a target object according to claim 1, characterized in that: Determining a first area from the plurality of target areas, the method further comprises: Determining, among the area types of the multiple target areas, an area type that matches a preset area type; The target region corresponding to the region type matching the preset region type is expanded and eroded by a morphological method to obtain the first region.

6. The method for determining a target object according to claim 1, characterized in that: After matching the type information of the first target object with preset target object type information, the method further includes: In the case that the type of the first target object does not match the preset target object type, the type of the first target object is sent to the target terminal, so that the target terminal determines again whether the first target object is the target object.

7. A device for determining a target object, characterized in that: include: A first acquisition module is used to perform semantic segmentation processing on the first image data through a semantic segmentation model to obtain multiple target regions of the first image data, and determine a first region from the multiple target regions; A second acquisition module, configured to perform instance segmentation processing on a first region in the first image data by using an instance segmentation model to obtain a first target object in the first region and type information of the first target object; a determination module, configured to match the type information of the first target object with preset target object type information, and determine that the first target object is a target object when the type information of the first target object matches the preset target object type information; Among them, the first acquisition module is also used to determine the global information of the first image data and the local information of the first image data through the semantic segmentation model; segment the multiple target areas in the first image data according to the local information, and determine the region types of the multiple target areas according to the global information, wherein the semantic segmentation model is a two-way semantic segmentation model.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the processor executes the method described in any one of claims 1 to 6 when running the program.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

Images