Alarm method and device based on regional video and program product

Abstract

The invention discloses an alarm method and device based on a regional video and a program product, and relates to the field of image processing. The method comprises the following steps: acquiring video data of a target area; image frames in the video data are input into a target deep learning model, an identification result of a target object in the image frames is output, the identification result of the target object comprises N object types and feature information of the target object, and the target deep learning model comprises a deep learning model trained based on a pixel size and a probability distribution model; determining a target type of the target object based on an identification result of the target object; and based on the target type of the target object, judging whether the target object meets the alarm condition of the target area to obtain a target judgment result, and generating target alarm information under the condition that the target judgment result indicates that the alarm condition of the target area is met. Through the method and the device, the technical problem of poor alarm effect caused by inaccurate detection result of the abnormal target in the detection area is solved.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular, to an alarm method, device, and program product based on regional video. Background Art

[0002] When detecting whether there are illegal behaviors in a target area (for example, illegal parking of motor vehicles, prohibited entry and placement of electric bicycles or motorcycles, etc.), the target detection technology in the related art has a low accuracy rate, is easily interfered, and the accuracy cannot be effectively guaranteed. For example, currently, methods based on machine learning or deep learning have poor discrimination effects on vehicles of different types but similar shapes or features, resulting in a high false alarm rate.

[0003] Aiming at the problem that the detection result of abnormal targets in the detection area in the related art is inaccurate, resulting in poor alarm effects, no effective solution has been proposed yet. Summary of the Invention

[0004] The main purpose of this application is to provide an alarm method, device, and program product based on regional video to solve the problem that the detection result of abnormal targets in the detection area in the related art is inaccurate, resulting in poor alarm effects.

[0005] To achieve the above object, according to one aspect of this application, an alarm method based on regional video is provided. The method includes: obtaining video data of a target area; inputting an image frame in the video data into a target deep learning model to output an identification result of a target object in the image frame, where the identification result of the target object includes: N types of object types and feature information of the target object, the target deep learning model includes: a deep learning model trained based on a pixel size and a probability distribution model, the probability distribution model is used to represent the distribution law of data, the N types of object types include at least one of the following: item type, species type, and N is an integer; based on the identification result of the target object, determining a target type of the target object, where the target type is one of the N types of object types; based on the target type of the target object, determining whether the target object meets the alarm condition of the target area to obtain a target judgment result, and generating a target alarm information when the target judgment result indicates that the target object meets the alarm condition of the target area.

[0006] Further, the target deep learning model is obtained through the following steps: obtaining original training samples, where the original training samples include images of P object types, and P is a positive integer greater than N; for the images in the original training samples, analyzing the number of images in different pixel size intervals associated with each object type to obtain an analysis result; constructing a probability distribution model, and based on the analysis result and the probability distribution model, updating the original training samples to obtain updated original training samples; training a deep learning model based on the updated original training samples to obtain the target deep learning model.

[0007] Further, updating the original training samples based on the analysis result and the probability distribution model to obtain updated original training samples includes: generating T target images based on the probability distribution model and the analysis result, so that the number of images in different pixel size intervals associated with each object type conforms to the distribution law of the probability distribution model, where T is a positive integer; adding the T target images and the object type of each target image to the original training samples to obtain the updated original training samples.

[0008] Further, the warning conditions for the target area include: a first warning condition and a second warning condition. The first warning condition includes that a preset type of object appears in the target area, and the second warning condition includes that the stationary duration of a preset type of object in the target area exceeds a preset duration. Based on the target type of the target object, determining whether the target object meets the warning conditions for the target area to obtain a target judgment result, including: based on the target type of the target object, determining whether the target object meets the first warning condition to obtain a first judgment result; in the case where the first judgment result indicates that the target object does not meet the first warning condition, determining that the target judgment result is that the target object does not meet the warning conditions for the target area; in the case where the first judgment result indicates that the target object meets the first warning condition, determining a preset threshold based on the pixels of the target object, and analyzing the motion state of the target object based on the preset threshold to obtain an analysis result, where the analysis result includes whether the target object is in a stationary state and the stationary duration when the target object is in a stationary state, and the preset threshold is used to analyze whether the target object is in a stationary state; based on the analysis result, determining whether the target object meets the second warning condition to obtain a second judgment result, and determining the first judgment result and the second judgment result as the target judgment result.

[0009] Further, in the case where the target judgment result indicates that the target object meets the warning condition of the target area, generating target warning information further includes: in the case where the target judgment result indicates that the target object meets the first warning condition, determining first warning information based on the color of the target object and the target type of the target object; in the case where the target judgment result indicates that the target object meets the second warning condition, determining the target warning information based on the analysis result and the first warning information; in the case where the target judgment result indicates that the target object does not meet the second warning condition, determining the first warning information as the target warning information.

[0010] Further, after inputting an image frame in the video data into a target deep learning model and outputting an identification result of a target object in the image frame, it further includes: determining the contour of the target object in the image frame to obtain a target contour; in the image frame, extracting the image of the inscribed rectangle part of the target contour to obtain a target image; splitting the target image and filtering out the image of a preset area according to a preset filtering strategy to obtain M image blocks, where the preset area includes: an area that interferes with the color of the target object, and M is a positive integer; determining the color of the target object based on the M image blocks.

[0011] Further, determining the color of the target object based on the M image blocks includes: determining the mean value of the color values of the RGB channels of the M image blocks to obtain a target color mean value; obtaining the color values of the RGB channels of a standard color to obtain a standard color value; calculating the difference between the target color mean value and the standard color value to obtain a calculation result; determining the color of the target object based on the calculation result.

[0012] Further, determining a preset threshold based on the pixels of the target object includes: determining the pixel width and pixel height of the target object in the image frame; calculating the product of the pixel width and a first preset value to obtain a first value; calculating the product of the pixel height and a second preset value to obtain a second value; comparing the first value and the second value to obtain a comparison result, and determining the preset threshold based on the comparison result.

[0013] To achieve the above object, according to another aspect of the present application, there is provided an alarm device based on regional video. The device includes: an acquisition unit for acquiring video data of a target area; a first processing unit for inputting an image frame in the video data into a target deep learning model and outputting an identification result of a target object in the image frame, wherein the identification result of the target object includes: N types of object types and feature information of the target object, and the target deep learning model includes: a deep learning model trained based on pixel size and probability distribution model, the probability distribution model being used to represent the distribution law of data, and the N types of object types include at least one of the following: item type, species type, and N is an integer; a first determination unit for determining a target type of the target object based on the identification result of the target object, wherein the target type is one of the N types of object types; a second processing unit for determining whether the target object meets the alarm condition of the target area based on the target type of the target object, obtaining a target judgment result, and generating a target alarm information when the target judgment result indicates that the target object meets the alarm condition of the target area.

[0014] Further, the target deep learning model is obtained through the following sub-units: an acquisition sub-unit for acquiring original training samples, wherein the original training samples include: images of P types of object types, and P is a positive integer greater than N; an analysis sub-unit for analyzing, for the images in the original training samples, the number of images in different pixel size intervals associated with each object type, to obtain an analysis result; a first processing sub-unit for constructing a probability distribution model, and updating the original training samples based on the analysis result and the probability distribution model to obtain updated original training samples; a training sub-unit for training a deep learning model based on the updated original training samples to obtain the target deep learning model.

[0015] Further, the first processing sub-unit includes: a processing module for generating T target images based on the probability distribution model and the analysis result, so that the number of images in different pixel size intervals associated with each object type conforms to the distribution law of the probability distribution model, wherein T is a positive integer; an adding module for adding the T target images and the object type of each target image to the original training samples to obtain the updated original training samples.

[0016] Further, the warning conditions for the target area include: a first warning condition and a second warning condition. Among them, the first warning condition includes: a preset type of object appears in the target area. The second warning condition includes: the stationary duration of the preset type of object in the target area exceeds a preset duration. The second processing unit includes: a judgment subunit, configured to judge whether the target object meets the first warning condition based on the target type of the target object, and obtain a first judgment result; a first determination subunit, configured to determine that the target judgment result is that the target object does not meet the warning condition of the target area when the first judgment result indicates that the target object does not meet the first warning condition; a first processing subunit, configured to determine a preset threshold based on the pixels of the target object and analyze the motion state of the target object based on the preset threshold to obtain an analysis result when the first judgment result indicates that the target object meets the first warning condition. The analysis result includes whether the target object is in a stationary state and the stationary duration when the target object is in a stationary state. The preset threshold is used to analyze whether the target object is in a stationary state; a second processing subunit, configured to judge whether the target object meets the second warning condition based on the analysis result to obtain a second judgment result, and determine the first judgment result and the second judgment result as the target judgment result.

[0017] Further, the second processing unit includes: a second determination subunit, configured to determine first warning information based on the color of the target object and the target type of the target object when the target judgment result indicates that the target object meets the first warning condition; a third determination subunit, configured to determine the target warning information based on the analysis result and the first warning information when the target judgment result indicates that the target object meets the second warning condition; a fourth determination subunit, configured to determine the first warning information as the target warning information when the target judgment result indicates that the target object does not meet the second warning condition.

[0018] Further, the warning device based on regional video further includes: a determination unit, which, after inputting an image frame in the video data into a target deep learning model and outputting an identification result of a target object in the image frame, further includes: a second determination unit, configured to determine the contour of the target object in the image frame to obtain a target contour; an extraction unit, configured to extract an image of an inscribed rectangle part of the target contour in the image frame to obtain a target image; a segmentation processing unit, configured to segment the target image and filter out images in a preset area according to a preset filtering strategy to obtain M image blocks, where the preset area includes: an area that interferes with the color of the target object, and M is a positive integer; a third determination unit, configured to determine the color of the target object based on the M image blocks.

[0019] Further, the third determination unit includes: an average value determination subunit, configured to determine the average value of the color values of the RGB channels of the M image blocks to obtain a target color average value; a color value acquisition subunit, configured to acquire the color values of the RGB channels of a standard color to obtain a standard color value; a calculation subunit, configured to calculate the difference between the target color average value and the standard color value to obtain a calculation result; a color determination subunit, configured to determine the color of the target object based on the calculation result.

[0020] Further, the first processing subunit includes: a determination module, configured to determine the pixel width and pixel height of the target object in the image frame; a first calculation module, configured to calculate the product of the pixel width and a first preset value to obtain a first numerical value; a second calculation module, configured to calculate the product of the pixel height and a second preset value to obtain a second numerical value; a comparison processing module, configured to compare the first numerical value and the second numerical value to obtain a comparison result, and determine the preset threshold based on the comparison result.

[0021] According to another aspect of the present application, there is provided a computer-readable storage medium, where the computer-readable storage medium includes a stored executable program, and when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the above-mentioned warning method based on regional video.

[0022] According to another aspect of the present application, there is provided an electronic device, including: a memory, storing an executable program; a processor, configured to run the program, where when the program runs, it executes the above-mentioned warning method based on regional video.

[0023] According to another aspect of the present application, there is provided a computer program product, including computer instructions, where when the computer instructions are executed by a processor, the steps of the above-mentioned warning method based on regional video are implemented.

[0024] In an embodiment of the present application, video data of a target area is acquired; an image frame in the video data is input into a target deep learning model, and an identification result of a target object in the image frame is output. The identification result of the target object includes: N types of object types and feature information of the target object. The target deep learning model includes: a deep learning model trained based on pixel size and a probability distribution model, and the probability distribution model is used to represent the distribution law of data. The N types of object types include at least one of the following: item type, species type, and N is an integer; based on the identification result of the target object, the target type of the target object is determined, where the target type is one of the N types of object types; based on the target type of the target object, it is determined whether the target object meets the alarm condition of the target area, and a target judgment result is obtained. When the target judgment result indicates that the target object meets the alarm condition of the target area, a target alarm message is generated, thereby solving the technical problem in the related art that the detection result of an abnormal target in a detection area is inaccurate, resulting in a poor alarm effect.

[0025] In the present invention, a target deep learning model trained based on a probability distribution model and pixel size identifies the type and feature information of a target object in a video, determines the true object type (i.e., the target type) of the target object according to the object type and feature information, and determines whether to meet the alarm condition based on the target type of the target object, and generates a target alarm message, avoiding the situation in the related art where the accuracy of the target detection technology is low, resulting in a high false alarm rate, thereby achieving the technical effects of improving the accuracy of the detection result of the target object in the area and reducing the false alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0027] Figure 1 A hardware structure block diagram of a computer terminal for implementing an alarm method based on area video is shown;

[0028] Figure 2 It is a flowchart of an alarm method based on area video according to an embodiment of the present application;

[0029] Figure 3 It is a schematic diagram of an alarm system based on area video according to an embodiment of the present application;

[0030] Figure 4 It is a flowchart of an alarm based on area video according to an embodiment of the present application;

[0031] Figure 5 It is a schematic diagram of an alarm device based on area video according to an embodiment of the present application;

[0032] Figure 6 It is a structural block diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0033] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order different from those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device including a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0035] It should be noted that the information collected in the present application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data and other processing all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set between this system and relevant users or institutions to provide corresponding operation entrances for users to choose to agree or refuse the automated decision-making results; if the user chooses to refuse, the expert decision-making process will be entered.

[0036] Embodiment 1

[0037] According to an embodiment of the present application, a method embodiment of an alarm method based on regional video is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0038] The method embodiment provided by the first embodiment of this application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Figure 1 The following shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an alarm method based on regional video. As Figure 1 shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b,..., 102n in the figure) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0039] It should be noted that the above one or more processors 102 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuit is a kind of processor control (such as the selection of a variable resistor terminal path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the alarm method based on regional video in the embodiments of this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned alarm method based on regional video. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] The display can be, for example, a touch-screen Liquid Crystal Display (LCD), which enables the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0043] Under the above operating environment, the present application provides an Figure 2 alarm method based on regional video as shown. Figure 2 It is a flowchart of the alarm method based on regional video according to Embodiment 1 of the present application.

[0044] Step S201, obtain video data of the target area.

[0045] The above video data may include: video data collected by video monitoring facilities in the target area, and the monitoring facilities may include, but are not limited to: visible light dome cameras, visible light bullet cameras, and brackets or bases for installing cameras, ranging devices, etc. The above video data may also include: video data obtained based on different visual sensors such as visible light, low light, infrared, and laser fill light.

[0046] Step S202, input the image frames in the video data into the target deep learning model, and output the recognition result of the target object in the image frames. Among them, the recognition result of the target object includes: N types of object types and the feature information of the target object. The target deep learning model includes: a deep learning model trained based on pixel size and probability distribution model, and the probability distribution model is used to represent the distribution law of data. The N types of object types include at least one of the following: item type, species type, and N is an integer.

[0047] The above-mentioned target deep learning model may include: a deep learning model trained based on sample annotation + sample target type statistics (by pixel size) + sample target quantity and ratio optimization (by pixel size). The above-mentioned target objects may include, but are not limited to: vehicles, markers, people, animals, etc. in the target area. The above-mentioned N types of object types may be the object types of the target objects predicted by the target deep learning model. The above-mentioned object types may include, but are not limited to: item types (e.g., vehicle types, traffic markers), species types (e.g., people, animal types). Vehicle types may include, but are not limited to: small passenger cars (sedans, SUVs (Sport Utility Vehicles), minivans, MPVs (Multi-Purpose Vehicles), scooters), medium and large passenger cars (buses, minibuses, coaches), transport vehicles (pickups, trucks, lorries, muck trucks), engineering vehicles (excavators, bulldozers, pile drivers, cranes, pump trucks, rollers, tractors, concrete mixers, forklifts), special vehicles (fire trucks, ambulances), enclosed vehicles (van trucks, tank trucks, fog cannon trucks, sprinkler trucks, garbage trucks, suction trucks), two-wheeled or three-wheeled vehicles (electric vehicles, motorcycles, bicycles, tricycles, small three-wheeled passenger vehicles). The above-mentioned traffic markers may include, but are not limited to: traffic cones, columns. The above-mentioned animal types may include, but are not limited to: people, animals, animal groups, cows, horses, donkeys, dogs, sheep, etc. The fine-grained distinction of object types reaches the specific types of specific vehicles or other objects. For example, taking small passenger cars as an example, the fine-grained distinction reaches specific types such as "sedans, SUVs, minivans, MPVs, scooters".

[0048] The feature information of the above-mentioned target objects may include: the feature parts of the target objects. For example, the feature parts of a vehicle. For example, the feature part of a fog cannon truck may be the sprayer at the rear of the vehicle body. The above-mentioned target deep learning model may be used to predict the object type of the target object and the feature information of the target object. For example, the information of the vehicle type and the feature parts of the vehicle.

[0049] Step S203: Based on the recognition result of the target object, determine the target type of the target object, where the target type is one of the N types of object types.

[0050] In this embodiment, the target type of the target object may be determined by combining the object class of the target object and the feature information of the target object. Taking the target object as a vehicle as an example, the following is an illustration:

[0051] For specific vehicle types, traffic markers, as well as people or various animals under the categories of small passenger cars, medium and large passenger cars, transport vehicles, engineering vehicles, special vehicles, etc., since the samples are rich, easy to collect, have obvious features, and are easy to distinguish from each other, the object types inferred by the deep learning model can be directly used as the target types to be output; for closed vehicles and two-wheeled or three-wheeled vehicles, since they are not easy to distinguish from each other. For example, a tanker truck transporting oil or hazardous chemicals and a fog cannon vehicle both have the front of a truck, a frame, and a closed tank-like container for storing liquids or gases in their main parts, making it extremely difficult to distinguish. Therefore, in this embodiment, a more effective strategy is provided, which may specifically include:

[0052] Predict the vehicle type (corresponding to the object type) + vehicle features (corresponding to the feature information) through the target deep learning module. Specifically, it may include: the target deep learning model simultaneously gives the vehicle type and the recognition result of a certain feature part of the vehicle, and then the vehicle type and the feature part can be used to comprehensively judge what type of vehicle it is.

[0053] Taking the fog cannon vehicle as an example, the typical feature part of the fog cannon vehicle is the sprayer at the rear of the vehicle body. If the target deep learning model simultaneously gives the recognition results of the fog cannon vehicle and the sprayer and the distance between the fog cannon vehicle and the sprayer does not exceed a certain threshold, then it is determined that the target object is a fog cannon vehicle; taking electric vehicles and motorcycles as an example, it is extremely difficult to distinguish between electric vehicles and motorcycles. The typical feature part of an electric vehicle is that there is no exhaust pipe on the lower right side of the vehicle body while a motorcycle has one. The colors and words of the license plate areas of electric vehicles and motorcycles are also different. Most electric vehicles have footrests while motorcycles do not. Electric vehicles do not have a fuel tank while motorcycles have a fuel tank and the fuel tank of a motorcycle may be in the front of the vehicle. If the target deep learning model simultaneously gives the recognition results of an electric vehicle or a motorcycle and these feature parts and the distance between the vehicle and the feature parts does not exceed a certain threshold, then electric vehicles and motorcycles can be accurately distinguished. This avoids the situation in related technologies where traditional object detection technologies based on region selection (sliding window), feature extraction, classifiers, etc. have low accuracy, are easily interfered with, and the accuracy cannot be effectively guaranteed. At the same time, it also solves the problem of poor recognition effect of small and micro targets in the solutions based on deep learning technologies in related technologies.

[0054] Step S204: Based on the target type of the target object, determine whether the target object meets the warning conditions of the target area, obtain a target judgment result, and generate a target warning message when the target judgment result indicates that the target object meets the warning conditions of the target area.

[0055] The above warning conditions may include: one or several set vehicles appear in the target area, and may also include: the stationary duration of the vehicle in the target area exceeds the preset duration. In this embodiment, based on the target type of the target object, it can be determined whether the target object meets the warning conditions of the target area to obtain a target judgment result. And when the target judgment result indicates that the target object meets the warning conditions of the target area, a target warning message is generated. The above target warning message may include, but is not limited to: the target type, the color of the target object, whether the target object is in a stationary state, etc. When the target judgment result indicates that the warning conditions of the target area are not met, there is no need to generate a warning message, that is, no warning is made.

[0056] A warning system based on regional video can be used to execute the warning method based on regional video provided in this embodiment. Figure 3 It is a schematic diagram of the warning system based on regional video provided in the embodiment of the present application, as Figure 3 shown, including: video monitoring facilities, a network, a server, and a client. The video monitoring facilities can collect video data in the target area, and then transmit the collected video data to the server through the network. The server can execute a warning algorithm to analyze the video data and generate an event warning message (corresponding to the target warning message), and then can send the event warning message to the client, and the client receives the information to obtain the specific information of the incident area and the abnormal event.

[0057] Optionally, in the warning method based on regional video provided in the embodiment of the present application, the target deep learning model is obtained in the following manner: Obtain original training samples, where the original training samples include: images of P object types, and P is a positive integer greater than N; for the images in the original training samples, analyze the number of images in different pixel size intervals associated with each object type to obtain an analysis result; construct a probability distribution model, and based on the analysis result and the probability distribution model, update the original training samples to obtain updated original training samples; train the deep learning model based on the updated original training samples to obtain the target deep learning model.

[0058] The above original training samples may include: images of P object types and an annotation document. The annotation document can record the annotation information of the images, and the annotation information may include: the object type and the information of the characteristic parts of the target (or called object) in the image. In this embodiment, the training labels of the deep learning model may be the object type and the information of the characteristic parts.

[0059] The above pixel size range may include: the range of the height of the image pixels and the width of the image pixels. In this embodiment, the instance segmentation deep learning model optimization for small and micro targets can specifically include: sample annotation + sample target type statistics (by size (corresponding to pixel size)) + sample target quantity and ratio optimization (by size), specifically including:

[0060] Step (1), annotate the original samples (including images of P object types) (for example, annotate vehicle types, traffic sign types, people or animals, and corresponding feature part information, etc.) to obtain the above original training samples;

[0061] Step (2), the number of images of different object types (i.e., sample labels) with different sizes ([1*1, 15*15), [15*15, 30*30), [30*30, 50*50), [50*50, 80*80), [80*80, w*h)), [a*b, c*d)) can be counted, and the statistical result is used as the above analysis result;

[0062] Step (3), in the above step (2), w is the width of the image pixels and h is the height of the image pixels;

[0063] Step (4), in the above step (2), [a*b, c*d) is a left-closed and right-open interval, and a and c are pixel widths, and b and d are pixel heights;

[0064] Step (5), in the above step (2), if the width of the target pixel size conforms to a certain size range 1 and the height of the target pixel size conforms to a certain size range 2, then the target pixel size conforms to the larger range of range 1 and range 2; for example, if a certain target pixel size is 14*16, then the pixel size of this target conforms to the range [15*15, 30*30);

[0065] Step (6), after completing the statistics in the above step (2), a distribution can be constructed. According to historical experience, the parameters of the probability distribution model (for example, μ and σ in the normal distribution 2 ) can be determined, and a probability distribution model (for example, normal distribution N(μ, σ 2 ) is constructed accordingly, where μ represents the mean and σ 2 represents the variance), so that a certain target X (images of the same object type but different sizes) conforms to the normal distribution N, where μ and σ can be determined in advance according to actual needs, and corresponding normal distributions can be constructed for all types of targets;

[0066] Step (7), after completing the above step (6), according to the above normal distribution, a method for generating targets, such as copy-paste, can be used to generate new target objects on the original training samples, so that the number of images in different pixel size intervals associated with each object type conforms to the distribution law of the probability distribution model, and the annotation information of the newly generated targets (i.e., the information of the object type and the feature parts) can also be added to the annotation document;

[0067] Step (8), after completing the above step (7), deep learning model training can be performed based on the finally obtained updated original training samples until the deep learning model meets the preset convergence conditions (for example, the recognition accuracy of the deep learning model reaches the preset conditions), and the target deep learning model is obtained.

[0068] In this embodiment, the data of images of different object types is statistically analyzed according to the pixel scale, the original training samples are updated according to the statistical results, and finally the deep learning model is trained based on the updated original training samples, achieving the purpose of improving the quality of the training samples, avoiding the problem of poor recognition effect of small and micro targets in the related art, and realizing the technical effect of improving the recognition accuracy of the finally obtained target deep learning model.

[0069] Optionally, in the warning method based on regional video provided in the embodiment of the present application, updating the original training samples based on the analysis result and the probability distribution model to obtain the updated original training samples includes: generating T target images based on the probability distribution model and the analysis result, so that the number of images in different pixel size intervals associated with each object type conforms to the distribution law of the probability distribution model, where T is a positive integer; adding the T target images and the object type of each target image to the original training samples to obtain the updated original training samples.

[0070] In this embodiment, according to the above normal distribution, a method for generating targets, such as copy-paste, is used to generate new target objects on the original training samples to obtain the updated original training samples, so that the number of images in different pixel size intervals associated with each object type conforms to the distribution law of the probability distribution model, and the annotation information of the newly generated targets (i.e., the information of the object type and the feature parts) can also be added to the annotation document.

[0071] For example, in this embodiment, the existing original training samples are analyzed to count the number of each target type (such as people, vehicles, animals, etc.) in different pixel size intervals. Then, a probability distribution model can be established according to the pre-set parameters (such as the parameters of the probability distribution model) (for example, normal distribution, Poisson distribution, uniform distribution, etc.). For a certain object type (such as people, vehicles, animals, etc.) whose number in different pixel size intervals does not conform to the probability distribution model, images of the same object type but different sizes can be generated using the original training samples by means of copy-paste and updated to the original training samples, so that the updated original training samples conform to the size interval probability distribution law described in the probability distribution model, that is, the pixel sizes of the images included in the generated images will appear according to the distribution law in the probability distribution model, balancing the number of samples of different target sizes in the training samples. Especially when there are few target samples of certain sizes in the training samples, by generating supplements, the situation that the recognition performance of the model for certain size targets is poor during training can be avoided, achieving the purpose of improving the generalization ability of the model and ensuring that the model has good recognition effects when facing targets of different sizes. Especially when dealing with tasks such as target detection and instance segmentation, it is particularly important to improve the recognition rate of small and micro targets.

[0072] Through the above steps, in this embodiment, the target deep learning model trained based on the probability distribution model and pixel size identifies the type and feature information of the target object in the video, determines the true object type (i.e., the target type) of the target object according to the object type and feature information, and judges whether it meets the alarm condition according to the target type of the target object, and generates target alarm information, avoiding the situation of low accuracy of the detection results of the target detection technology in the related art, resulting in a high false alarm rate, thereby achieving the technical effect of improving the accuracy of the detection results of the target objects in the area and reducing the false alarm rate, and further solving the technical problem of inaccurate detection results of abnormal targets in the detection area in the related art, resulting in poor alarm effects.

[0073] Optionally, in the warning method based on regional video provided in the embodiments of the present application, the warning conditions for the target area include: a first warning condition and a second warning condition. Among them, the first warning condition includes: a preset type of object appears in the target area. The second warning condition includes: the stationary duration of the preset type of object in the target area exceeds a preset duration. Based on the target type of the target object, it is determined whether the target object meets the warning conditions of the target area, and a target judgment result is obtained, including: based on the target type of the target object, it is determined whether the target object meets the first warning condition, and a first judgment result is obtained; in the case where the first judgment result indicates that the target object does not meet the first warning condition, it is determined that the target judgment result is that the target object does not meet the warning conditions of the target area; in the case where the first judgment result indicates that the target object meets the first warning condition, a preset threshold is determined based on the pixels of the target object, and the motion state of the target object is analyzed based on the preset threshold, and an analysis result is obtained. Among them, the analysis result includes: whether the target object is in a stationary state and the stationary duration when the target object is in a stationary state. The preset threshold is used to analyze whether the target object is in a stationary state; based on the analysis result, it is determined whether the target object meets the second warning condition, and a second judgment result is obtained, and the first judgment result and the second judgment result are determined as the target judgment result.

[0074] The warning conditions for the above-mentioned target area include but are not limited to: a first warning condition and a second warning condition. Among them, the first warning condition may include: a preset type of object appears in the target area. For example, if the target area is an area where a certain type of vehicle is prohibited from parking, then the first warning condition may be set as "the vehicle of this type appears in the target area"; or, when one or more of the following targets appear in the set specific detection area (ROI) (corresponding to the target area): small passenger cars (sedans, SUVs, minivans, MPVs, scooters), medium and large passenger cars (buses, minibuses, buses), transport vehicles (pickups, trucks, trucks, muck trucks), engineering vehicles (excavators, bulldozers, pile drivers, cranes, pump trucks, rollers, tractors, concrete trucks, forklifts), special vehicles (fire trucks, ambulances), enclosed vehicles (van trucks, tank trucks, fog cannon trucks, sprinkler trucks, garbage trucks, vacuum cleaners), two-wheeled or three-wheeled vehicles (electric vehicles, motorcycles, bicycles, tricycles, manned tricycles), traffic markers (traffic cones, columns), people or animals (people, animals, animal groups, cows, horses, donkeys, dogs, sheep).

[0075] It should be noted that when the first warning condition (or warning condition 1) is met, a warning can be issued, and comprehensive warning 1 can be carried out by integrating information such as the target type and vehicle color (if the target is a vehicle, the vehicle color can be calculated). When the first warning condition is not met, a warning may not be issued. At this time, the target judgment result may be that the target object does not meet the warning conditions of the target area.

[0076] The second warning condition can focus on the motion state of the object. Specifically, for an object of a preset type (e.g., a vehicle) within the target area, if its stationary duration exceeds a preset duration, the minimum value of the pixel width of the target ID (corresponding to the target object) * coefficient 1 and the pixel height of the target ID * coefficient 2 is used as the judgment threshold for the target ID (e.g., the vehicle) being stationary (corresponding to the preset threshold). It can be judged by the relationship between the position difference of the center points of the target ID in two adjacent frames processed by the algorithm and a preset position difference threshold. When the position difference is less than the preset position difference threshold, it is in a stationary state; otherwise, it is in a motion state. Track the target ID in the video, and count the duration of the stationary ID. When the stationary duration of the stationary ID reaches the stay duration threshold (i.e., the preset duration threshold), it is determined that the target object is abnormal (e.g., the vehicle is abnormal (suspected of illegal parking, vehicle accident, or vehicle failure)).

[0077] For example, based on a preset threshold, analyze the motion state of the target object to obtain an analysis result, which may include: using a target tracking algorithm to identify a video stream of a certain duration within the target area (which can be the above-mentioned video data), and completing the analysis of vehicle stillness based on this video stream. If the stationary duration of the target object (e.g., a vehicle) reaches the preset duration threshold (which can be customized), it is determined that the target object is abnormal (e.g., suspected of illegal parking, vehicle accident, or vehicle failure).

[0078] The specific method for analyzing the motion state can be: track the target ID (corresponding to the target object) in the video stream, count the duration of the stationary ID. When the stationary duration of the stationary ID reaches the preset duration threshold, the warning condition 2 (the second warning condition) is satisfied; during the process of counting the stationary duration of a certain stationary ID, if it is in a motion state in a new frame, the stationary duration of this ID is cleared and recalculated.

[0079] The method for judging whether the motion state is stationary or moving can include: judging by the relationship between the position difference of the center points of the target ID in two adjacent frames processed by the algorithm (there may be intervening frames between the two algorithm-processed frames, and the number of intervening frames = 0, 1, 2, 3,..., which can be set according to actual needs) and a preset position difference threshold. When the position difference is less than the preset position difference threshold, it is in a stationary state; otherwise, it is in a motion state.

[0080] It should be noted that when the second warning condition (or warning condition 2) is satisfied, for example, the vehicle has been stationary for a certain duration, an alarm can be issued. At this time, the target judgment result can be that the first warning condition and the second warning condition are satisfied. Moreover, in this embodiment, the information of the comprehensive alarm 1 and the stationary duration information of the target object can also be combined to perform a comprehensive alarm 2. When the second warning condition is not satisfied, the target judgment result is that the first warning condition is satisfied and the second warning condition is not satisfied.

[0081] By determining whether the target object in the target area meets the alarm condition, the accuracy and timeliness of the alarm are ensured, unnecessary false alarms are avoided, and at the same time, the situations that need to be alarmed can be quickly identified, thereby improving the efficiency and effectiveness of the video monitoring system.

[0082] Optionally, in the alarm method based on regional video provided in the embodiments of the present application, when the target judgment result indicates that the target object meets the alarm condition of the target area, generating target alarm information further includes: when the target judgment result indicates that the target object meets the first alarm condition, determining the first alarm information based on the color of the target object and the target type of the target object; when the target judgment result indicates that the target object meets the second alarm condition, determining the target alarm information based on the analysis result and the first alarm information; when the target judgment result indicates that the target object does not meet the second alarm condition, determining the first alarm information as the target alarm information.

[0083] In this embodiment, when the target judgment result indicates that the target object meets the first alarm condition (i.e., alarm condition 1), information such as the target type, vehicle color (if the target object is a vehicle, the vehicle color can be calculated), etc. (corresponding to the first alarm information) can be integrated to perform comprehensive alarm 1.

[0084] When the target judgment result indicates that the target object meets the second alarm condition, the target alarm information can be determined based on the analysis result and the first alarm information. For example, the target alarm information can be obtained by integrating the first alarm information and the stationary duration of the target object in the analysis result.

[0085] When the target judgment result indicates that the target object does not meet the second alarm condition, the first alarm information can be directly used as the target alarm information.

[0086] According to the alarm condition met by the target object indicated by the target judgment result, the corresponding target alarm information is obtained, ensuring the accuracy and timeliness of the alarm, avoiding unnecessary false alarms, and at the same time being able to quickly identify the situations that need to be alarmed, thereby improving the efficiency and effectiveness of the video monitoring system.

[0087] Optionally, in the warning method based on regional video provided in the embodiments of the present application, after inputting the image frames in the video data into the target deep learning model and outputting the recognition result of the target object in the image frames, the method further includes: determining the contour of the target object in the image frame to obtain a target contour; extracting the image of the inscribed rectangle part of the target contour in the image frame to obtain a target image; segmenting the target image and filtering out the images in the preset area according to a preset filtering strategy to obtain M image blocks, where the preset area includes: the area that interferes with the color of the target object, and M is a positive integer; determining the color of the target object based on the M image blocks.

[0088] (1) In this embodiment, according to the result of instance segmentation (assuming that the target depth model infers N targets, that is, Ti, i = 1,..., N), for each target (corresponding to the target object), the target pixels can be extracted along the contour edge.

[0089] (2) In this embodiment, the inscribed rectangle of the target contour can be calculated, and the target pixels in the inscribed rectangle part can be extracted.

[0090] (3) For the local image Si (corresponding to the target image) composed of the target pixels in the inscribed rectangle part, where i = 1,..., N, if it is segmented r times by rows and c times by columns, then Si is divided into M' (M' = (r + 1)*(c + 1)) image blocks Bu,v, u = 1,..., r + 1; v = 1,..., c + 1.

[0091] (4) Bu,v (u = floor(0.5*(r + 1)),..., floor(0.8*(r + 1)), v = floor(0.2*(c + 1)),..., floor(0.8*(c + 1))) can be taken, a total of M image blocks, where M = (floor(0.8*(r + 1)) - floor(0.5*(r + 1)))*(floor(0.8*(c + 1)) - floor(0.2*(c + 1))), where floor() represents taking the integer part. The values of u and v in Bu,v can exclude the areas that interfere with the color of the target object. For example, there are usually windows on the upper part of the vehicle that occupy a large image area, and the color of the windows will seriously interfere with the calculation of the vehicle color; there are usually tires on the lower part of the vehicle near the lower edge in contact with the ground that occupy a large image area, and the color of the tires will seriously interfere with the calculation of the vehicle color; in addition, the front and rear of the vehicle are also prone to color interference. Therefore, these several color interferences can be excluded to improve the accuracy of the vehicle color recognition result.

[0092] After splitting to obtain M image patches, the color of the target object can be determined based on the colors of the M image patches, solving the problems in the related art that directly extracting the images of the vehicle area for vehicle color calculation is vulnerable to the interference of the image pixels of the window, tire, front of the vehicle, and rear of the vehicle, with large errors, and for vehicle color classification based on machine learning algorithms, the accuracy is limited, the generalization ability is insufficient, and for color classification based on deep learning, a large number of samples are required, and both the training and deployment of the deep learning model require computing resources and the generalization ability is insufficient, achieving the technical effect of improving the accuracy of the vehicle color recognition result.

[0093] Optionally, in the warning method based on regional video provided in the embodiments of the present application, determining the color of the target object based on the M image patches includes: determining the mean value of the color values of the RGB channels of the M image patches to obtain the target color mean value; obtaining the color values of the RGB channels of the standard color to obtain the standard color value; calculating the difference between the target color mean value and the standard color value to obtain the calculation result; and determining the color of the target object based on the calculation result.

[0094] In this embodiment, determining the mean value of the color values of the RGB channels of the M image patches to obtain the target color mean value; obtaining the color values of the RGB channels of the standard color to obtain the standard color value, and then the difference calculation can be performed between the RGB mean value of the image patch (i.e., the target color mean value) and the RGB value of the standard color (corresponding to the standard color value), and the standard color with the smallest difference is used as the color of the target object.

[0095] For example, first, the color means of the RGB channels of the M Bu, v image patches can be calculated respectively;

[0096] Secondly, the color means of the RGB channels of the M Bu, v image patches (corresponding to the target color mean value) are used as the RGB color values of Si (corresponding to the target object);

[0097] In addition, the difference can be calculated between the RGB color value of Si and the RGB values of standard colors such as red, yellow, green, cyan, blue, purple, pink, brown, white, gray, black, silver, orange, etc., that is, the absolute value differences between the R, G, B of Si and the RGB of the standard color are calculated respectively;

[0098] Finally, if the difference between Si and a certain color (Ck) in the standard color is the smallest, then the color of Si is Ck, that is, the color of the target object (for example, the vehicle) is Ck.

[0099] Determining the color of the target object according to the mean value of the RGB channel colors of the M image patches solves the problems of easy interference, low accuracy, insufficient generalization ability, high computing power requirements, etc. in vehicle color calculation. Thus, the technical effect of improving the accuracy of the vehicle color recognition result is achieved.

[0100] Optionally, in the warning method based on regional video provided in the embodiments of the present application, determining a preset threshold based on the pixels of the target object includes: determining the pixel width and pixel height of the target object in the image frame; calculating the product of the pixel width and a first preset value to obtain a first value; calculating the product of the pixel height and a second preset value to obtain a second value; comparing the first value and the second value to obtain a comparison result, and determining the preset threshold based on the comparison result.

[0101] In this embodiment, the minimum value of the pixel width of the target ID (corresponding to the target object) * coefficient 1 (corresponding to the first preset value) and the pixel height of the target ID * coefficient 2 (corresponding to the second preset value) can be used as the judgment threshold for judging the stationary state (corresponding to the preset threshold), that is, the minimum value of the first value and the second value can be determined as the preset threshold.

[0102] For example, preset threshold = min(pixel width of target ID * coefficient 1, pixel height of target ID * coefficient 2); where coefficient 1 and coefficient 2 can take 0.03.

[0103] Determining the preset threshold for analyzing the motion state of the target object according to the pixel width and pixel height of the target object achieves the purpose of adaptively setting the threshold for motion state analysis according to the pixel information of the target object, avoiding the situation that the analysis result of the motion state is inaccurate due to setting a fixed threshold, and realizing the technical effect of improving the accuracy of the vehicle motion state analysis result.

[0104] Figure 4 It is a warning flow chart based on regional video provided in the embodiments of the present application, as Figure 4 shown, including: obtaining a video image frame, instance segmentation (determining the target type of the target object in the image frame and performing vehicle color recognition), determining whether the warning conditions are met (warning condition 1 (corresponding to the first warning condition) and warning condition 2 (corresponding to the second warning condition)) and motion state analysis. When warning condition 1 is met, motion state analysis can be performed to analyze whether the target object is in a stationary state to obtain an analysis result, and it is determined whether warning condition 2 is met according to the analysis result. When warning condition 2 is met, warning 1 (corresponding to the first warning message) and the stationary duration of the target object can be integrated to obtain warning 2.

[0105] Through the warning method based on regional video provided in this embodiment, during the model training process, the recognition of small and micro targets is optimized through sample annotation + sample target type statistics (by size) + sample target quantity and ratio optimization (by size). By counting the quantities of different target types (i.e., sample labels) with different sizes ([1*1, 15*15), [15*15, 30*30), [30*30, 50*50), [50*50, 80*80), [80*80, w*h)); for a certain type of target (objects of a certain object type), a probability distribution model is constructed (for example, normal distribution N(μ, σ 2 ))), that is, it is assumed that a certain target X (images of the same object type but different sizes) conforms to the distribution law of the probability distribution model; probability distribution models are constructed for all object types of targets; according to the probability distribution model, a method of target generation, such as copy-paste, is used to generate new target objects on the original sample, and the annotation information of the newly generated targets is added to the annotation document; it can avoid the problems of low accuracy based on traditional target detection techniques and poor recognition effect of small and micro targets based on deep learning techniques.

[0106] During the vehicle type recognition process, the object type (for example, vehicle type) and the information of specific feature parts recognized by the target deep learning model are used to determine the target type of the target object according to the object type and feature parts, avoiding the problems of being easily interfered and having a high false alarm rate in the vehicle type judgment method, and achieving the technical effect of improving the accuracy of the vehicle type recognition result.

[0107] When analyzing the motion state of a vehicle, the minimum value of the pixel width of the target ID (corresponding to the target object, for example, the vehicle) * coefficient 1 and the pixel height of the target ID * coefficient 2 can be used as the judgment threshold for the vehicle to be stationary (corresponding to the preset threshold); the relationship between the position difference of the center points of the target IDs processed by the adjacent two algorithms and the preset position difference threshold is used for judgment. When the position difference is less than the preset position difference threshold, it is in a stationary state, otherwise it is in a moving state; the target IDs in the video are tracked, and the duration of the stationary IDs is counted. When the stationary duration of the stationary ID reaches the stay duration threshold (corresponding to the preset duration threshold), it is determined that the vehicle is abnormal (there is a suspicion of vehicle illegal parking, vehicle accident or vehicle failure), solving the problem that the vehicle stationary judgment method in the related technology is easily interfered and results in a high false alarm rate, and achieving the effects of improving the accuracy of the vehicle motion state judgment result and reducing the false alarm rate.

[0108] In the process of identifying the vehicle color, calculate the inscribed rectangle of the target contour of the target object (e.g., the vehicle) in the video frame, extract the target pixels of the inscribed rectangle part, and for the local image composed of the target pixels of the inscribed rectangle part, cut it into several image blocks row by row and column by column; exclude the window, tire, front of the vehicle, and rear of the vehicle parts in the image blocks, and calculate the RGB mean value of the remaining image blocks; calculate the difference between the RGB mean value of the image blocks and the RGB values of the standard colors, and take the standard color with the smallest difference as the color of the target, which solves the problems in the related art such as the vehicle color calculation being vulnerable to interference, low accuracy, insufficient generalization ability, and high computing power requirements, and realizes the technical effect of improving the accuracy of the vehicle color recognition result.

[0109] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0110] Embodiment 2

[0111] The embodiment of the present application also provides an alarm device based on regional video. It should be noted that the alarm device based on regional video in the embodiment of the present application can be used to execute the alarm method based on regional video provided by the embodiment of the present application. The alarm device based on regional video provided by the embodiment of the present application is introduced below.

[0112] According to the embodiment of the present application, there is also provided a device for implementing the above-mentioned alarm method based on regional video, as Figure 5 shown, the device includes: an acquisition unit 51, a first processing unit 52, a first determination unit 53, and a second processing unit 54.

[0113] Among them, the acquisition unit 51 is used to acquire video data of the target area;

[0114] The first processing unit 52 is used to input the image frame in the video data into the target deep learning model and output the recognition result of the target object in the image frame. Among them, the recognition result of the target object includes: N types of object types and the feature information of the target object. The target deep learning model includes: a deep learning model trained based on the pixel size and probability distribution model, and the probability distribution model is used to represent the distribution law of the data. The N types of object types include at least one of the following: item type, species type, and N is an integer;

[0115] The first determination unit 53 is used to determine the target type of the target object based on the recognition result of the target object, where the target type is one of the N types of object types;

[0116] A second processing unit 54, configured to determine whether the target object meets the alarm condition of the target area based on the target type of the target object, obtain a target judgment result, and generate a target alarm message when the target judgment result indicates that the target object meets the alarm condition of the target area.

[0117] In the alarm device based on regional video provided in the embodiment of the present invention, a obtaining unit 51 may be configured to obtain video data of a target area; a first processing unit 52 may be configured to input an image frame in the video data into a target deep learning model, and output an identification result of the target object in the image frame, where the identification result of the target object includes: N types of object types and feature information of the target object, the target deep learning model includes: a deep learning model trained based on a pixel size and a probability distribution model, the probability distribution model is used to represent the distribution law of data, the N types of object types include at least one of the following: item type, species type, and N is an integer; a first determination unit 53 may be configured to determine the target type of the target object based on the identification result of the target object, where the target type is one of the N types of object types; a second processing unit 54 may be configured to determine whether the target object meets the alarm condition of the target area based on the target type of the target object, obtain a target judgment result, and generate a target alarm message when the target judgment result indicates that the target object meets the alarm condition of the target area. Furthermore, the technical problem that the detection result of an abnormal target in a detection area in the related art is inaccurate, resulting in a poor alarm effect, is solved.

[0118] In this embodiment, the target deep learning model trained based on the probability distribution model and the pixel size identifies the type and feature information of the target object in the video, determines the true object type (i.e., the target type) of the target object according to the object type and the feature information, and determines whether it meets the alarm condition according to the target type of the target object, and generates a target alarm message, avoiding the situation of a low detection result accuracy and a high false alarm rate in the target detection technology in the related art, thereby achieving the technical effects of improving the accuracy of the detection result of the target object in the area and reducing the false alarm rate.

[0119] Optionally, in the alarm device based on regional video provided in the embodiment of the present application, the target deep learning model is obtained through the following sub-units: an obtaining sub-unit, configured to obtain original training samples, where the original training samples include: images of P types of object types, and P is a positive integer greater than N; an analysis sub-unit, configured to analyze the number of images in different pixel size intervals associated with each object type for the images in the original training samples, and obtain an analysis result; a first processing sub-unit, configured to construct a probability distribution model, update the original training samples based on the analysis result and the probability distribution model, and obtain updated original training samples; a training sub-unit, configured to train a deep learning model based on the updated original training samples to obtain the target deep learning model.

[0120] Optionally, in the warning device based on regional video provided in the embodiments of the present application, the first processing subunit includes: a processing module, configured to generate T target images based on a probability distribution model and an analysis result, so that the number of images in different pixel size intervals associated with each object type conforms to the distribution law of the probability distribution model, where T is a positive integer; an adding module, configured to add the T target images and the object type of each target image to the original training sample to obtain an updated original training sample.

[0121] Optionally, in the warning device based on regional video provided in the embodiments of the present application, the warning conditions of the target area include: a first warning condition and a second warning condition, where the first warning condition includes: an object of a preset type appears in the target area, and the second warning condition includes: the stationary duration of the object of the preset type in the target area exceeds a preset duration. The second processing unit includes: a judging subunit, configured to judge whether the target object meets the first warning condition based on the target type of the target object to obtain a first judgment result; a first determining subunit, configured to determine that the target judgment result is that the target object does not meet the warning conditions of the target area when the first judgment result indicates that the target object does not meet the first warning condition; a first processing subunit, configured to determine a preset threshold based on the pixels of the target object and analyze the motion state of the target object based on the preset threshold to obtain an analysis result when the first judgment result indicates that the target object meets the first warning condition, where the analysis result includes: whether the target object is in a stationary state and the stationary duration when the target object is in a stationary state, and the preset threshold is used to analyze whether the target object is in a stationary state; a second processing subunit, configured to judge whether the target object meets the second warning condition based on the analysis result to obtain a second judgment result, and determine the first judgment result and the second judgment result as the target judgment result.

[0122] Optionally, in the warning device based on regional video provided in the embodiments of the present application, the second processing unit includes: a second determining subunit, configured to determine a first warning message based on the color of the target object and the target type of the target object when the target judgment result indicates that the target object meets the first warning condition; a third determining subunit, configured to determine a target warning message based on the analysis result and the first warning message when the target judgment result indicates that the target object meets the second warning condition; a fourth determining subunit, configured to determine the first warning message as the target warning message when the target judgment result indicates that the target object does not meet the second warning condition.

[0123] Optionally, in the warning device based on regional video provided in the embodiments of the present application, the warning device based on regional video further includes: a determination unit, configured to, after inputting an image frame in video data into a target deep learning model and outputting an identification result of a target object in the image frame, further include: a second determination unit, configured to determine the contour of the target object in the image frame to obtain a target contour; an extraction unit, configured to extract an image of an inscribed rectangle part of the target contour in the image frame to obtain a target image; a segmentation processing unit, configured to segment the target image and filter out images in a preset area according to a preset filtering policy to obtain M image blocks, where the preset area includes: an area that interferes with the color of the target object, and M is a positive integer; a third determination unit, configured to determine the color of the target object based on the M image blocks.

[0124] Optionally, in the warning device based on regional video provided in the embodiments of the present application, the third determination unit includes: an average determination subunit, configured to determine the average value of the color values of the RGB channels of the M image blocks to obtain a target color average value; a color value acquisition subunit, configured to acquire the color values of the RGB channels of a standard color to obtain a standard color value; a calculation subunit, configured to calculate the difference between the target color average value and the standard color value to obtain a calculation result; a color determination subunit, configured to determine the color of the target object based on the calculation result.

[0125] Optionally, in the warning device based on regional video provided in the embodiments of the present application, the first processing subunit includes: a determination module, configured to determine the pixel width and pixel height of the target object in the image frame; a first calculation module, configured to calculate the product of the pixel width and a first preset value to obtain a first numerical value; a second calculation module, configured to calculate the product of the pixel height and a second preset value to obtain a second numerical value; a comparison processing module, configured to compare the first numerical value and the second numerical value to obtain a comparison result, and determine a preset threshold based on the comparison result.

[0126] It should be noted here that the above-mentioned acquisition unit 51, first processing unit 52, first determination unit 53, and second processing unit 54 correspond to steps S201 to S204 in Embodiment 1. The instances and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b,..., 102n), and the above-mentioned modules may also be part of the device and can run in the computer terminal 10 provided in Embodiment 1.

[0127] Embodiment 3

[0128] Embodiments of the present application may provide an electronic deviceFigure 6 is a block diagram of an electronic device according to an embodiment of the present application. As Figure 6 shown, the electronic device may include: one or more ( Figure 6 only one is shown in the figure) processors 602, a memory 604, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0129] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the above-mentioned method is implemented. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely set relative to the processor, and these remote memories can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] The processor can call the information and application programs stored in the memory through a transmission device to execute the following steps: obtaining video data of a target area; inputting an image frame in the video data into a target deep learning model, and outputting an identification result of a target object in the image frame, wherein the identification result of the target object includes: N types of object types and feature information of the target object, the target deep learning model includes: a deep learning model trained based on a pixel size and a probability distribution model, the probability distribution model is used to represent the distribution law of data, the N types of object types include at least one of the following: item type, species type, and N is an integer; based on the identification result of the target object, determining a target type of the target object, wherein the target type is one of the N types of object types; based on the target type of the target object, determining whether the target object meets an alarm condition of the target area, obtaining a target judgment result, and generating a target alarm information when the target judgment result indicates that the target object meets the alarm condition of the target area.

[0131] The processor can call the information and application programs stored in the memory through a transmission device to execute the following steps: the target deep learning model is obtained in the following manner: obtaining original training samples, wherein the original training samples include: images of P types of object types, and P is a positive integer greater than N; for the images in the original training samples, analyzing the number of images in different pixel size intervals associated with each object type, obtaining an analysis result; constructing a probability distribution model, and updating the original training samples based on the analysis result and the probability distribution model to obtain updated original training samples; training a deep learning model based on the updated original training samples to obtain the target deep learning model.

[0132] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: based on the analysis result and the probability distribution model, update the original training samples to obtain the updated original training samples, including: based on the probability distribution model and the analysis result, generate T target images, so that the number of images in different pixel size intervals associated with each object type conforms to the distribution law of the probability distribution model, where T is a positive integer; add the T target images and the object type of each target image to the original training samples to obtain the updated original training samples.

[0133] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: the alarm conditions for the target area include: the first alarm condition and the second alarm condition, where the first alarm condition includes: a preset type of object appears in the target area, and the second alarm condition includes: the stationary duration of a preset type of object in the target area exceeds a preset duration. Based on the target type of the target object, determine whether the target object meets the alarm conditions for the target area to obtain a target judgment result, including: based on the target type of the target object, determine whether the target object meets the first alarm condition to obtain a first judgment result; in the case where the first judgment result indicates that the target object does not meet the first alarm condition, determine that the target judgment result is that the target object does not meet the alarm conditions for the target area; in the case where the first judgment result indicates that the target object meets the first alarm condition, based on the pixels of the target object, determine a preset threshold, and based on the preset threshold, analyze the motion state of the target object to obtain an analysis result, where the analysis result includes: whether the target object is in a stationary state and the stationary duration when the target object is in a stationary state, and the preset threshold is used to analyze whether the target object is in a stationary state; based on the analysis result, determine whether the target object meets the second alarm condition to obtain a second judgment result, and determine the first judgment result and the second judgment result as the target judgment result.

[0134] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: in the alarm method based on regional video provided in the embodiments of the present application, in the case where the target judgment result indicates that the target object meets the alarm conditions for the target area, generate target alarm information, further including: in the case where the target judgment result indicates that the target object meets the first alarm condition, based on the color of the target object and the target type of the target object, determine the first alarm information; in the case where the target judgment result indicates that the target object meets the second alarm condition, based on the analysis result and the first alarm information, determine the target alarm information; in the case where the target judgment result indicates that the target object does not meet the second alarm condition, determine the first alarm information as the target alarm information.

[0135] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: After inputting the image frames in the video data into the target deep learning model and outputting the recognition results of the target objects in the image frames, it further includes: determining the contour of the target object in the image frame to obtain the target contour; extracting the image of the inscribed rectangle part of the target contour in the image frame to obtain the target image; segmenting the target image and filtering out the images in the preset area according to the preset filtering strategy to obtain M image blocks, where the preset area includes: the area that interferes with the color of the target object, and M is a positive integer; determining the color of the target object based on the M image blocks.

[0136] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: Determining the color of the target object based on the M image blocks includes: determining the average value of the color values of the RGB channels of the M image blocks to obtain the target color average value; obtaining the color values of the RGB channels of the standard color to obtain the standard color value; calculating the difference between the target color average value and the standard color value to obtain the calculation result; determining the color of the target object based on the calculation result.

[0137] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: Determining the preset threshold based on the pixels of the target object includes: determining the pixel width and pixel height of the target object in the image frame; calculating the product of the pixel width and the first preset value to obtain the first value; calculating the product of the pixel height and the second preset value to obtain the second value; comparing the first value and the second value to obtain the comparison result, and determining the preset threshold based on the comparison result.

[0138] By adopting the embodiments of the present application, the target deep learning model trained based on the probability distribution model and pixel size is used to identify the type and feature information of the target object in the video, determine the true object type (i.e., the target type) of the target object according to the object type and feature information, and judge whether it meets the alarm condition according to the target type of the target object, and generate the target alarm information, avoiding the situation in the related art where the detection result accuracy of the target detection technology is low, resulting in a high false alarm rate, thus achieving the technical effects of improving the accuracy of the detection results of the target objects in the area and reducing the false alarm rate. Furthermore, it solves the technical problem in the related art that the detection results of the abnormal targets in the detection area are inaccurate, resulting in a poor alarm effect.

[0139] Those of ordinary skill in the art can understand that Figure 6 The structure shown is only schematic, and the electronic device can also be a terminal device such as a smart phone, a tablet computer, a handheld computer, a Mobile Internet Device (MID), a PAD, etc. Figure 6It does not limit the structure of the above-mentioned electronic device. For example, the electronic device may further include more or fewer components (such as network interfaces, display devices, etc.) than those shown in Figure 6 or have a different configuration from that shown in Figure 6 .

[0140] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium. The storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0141] Embodiment 4

[0142] An embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the above storage medium may be used to store the program code executed by the warning method based on regional video provided in the first embodiment above.

[0143] Optionally, in this embodiment, the above storage medium may be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0144] The present application also provides a computer program product, which is suitable for executing a program for the steps of the warning method based on regional video when executed on a data processing device.

[0145] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0146] In the above embodiments of the present application, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0147] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other may be through some interfaces, and the indirect coupling or communication connection of the units or modules may be in an electrical or other form.

[0148] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] In addition, each functional unit in various embodiments of the present application may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0150] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: USB flash drive, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disc and other various media that can store program codes.

[0151] The above is only the preferred embodiment of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. An alarm method based on regional video, characterized in that, Including: Obtain video data of a target area; Input an image frame in the video data into a target deep learning model, and output an identification result of a target object in the image frame. Among them, the identification result of the target object includes: N types of object types and feature information of the target object. The target deep learning model includes: a deep learning model trained based on pixel size and probability distribution model. The probability distribution model is used to represent the distribution law of data. The N types of object types include at least one of the following: item type, species type, and N is an integer; Based on the identification result of the target object, determine the target type of the target object, where the target type is one of the N types of object types; Based on the target type of the target object, determine whether the target object meets the alarm condition of the target area, obtain a target judgment result, and generate a target alarm information when the target judgment result indicates that the target object meets the alarm condition of the target area.

2. The alarm method based on regional video according to claim 1, characterized in that The target deep learning model is obtained through the following method: Obtain original training samples, where the original training samples include images of P types of object types, P is a positive integer greater than N; For the images in the original training samples, analyze the number of images in different pixel size intervals associated with each object type to obtain an analysis result; Construct a probability distribution model, and update the original training samples based on the analysis result and the probability distribution model to obtain updated original training samples; Train a deep learning model based on the updated original training samples to obtain the target deep learning model.

3. The warning method based on regional video according to claim 2, characterized in that, Updating the original training samples based on the analysis result and the probability distribution model to obtain updated original training samples includes: Generate T target images based on the probability distribution model and the analysis result, so that the number of images in different pixel size intervals associated with each object type conforms to the distribution law of the probability distribution model, where T is a positive integer; Add the T target images and the object type of each target image to the original training samples to obtain the updated original training samples.

4. The warning method based on regional video according to claim 1, wherein The alarm conditions of the target area include: a first alarm condition and a second alarm condition. Among them, the first alarm condition includes: a preset type of object appears in the target area, and the second alarm condition includes: the stationary duration of a preset type of object in the target area exceeds a preset duration. Based on the target type of the target object, determine whether the target object meets the alarm condition of the target area, and obtain a target judgment result, including: Based on the target type of the target object, determine whether the target object meets the first alarm condition to obtain a first judgment result; When the first judgment result indicates that the target object does not meet the first alarm condition, determine that the target judgment result is that the target object does not meet the alarm condition of the target area; When the first judgment result indicates that the target object meets the first warning condition, based on the pixels of the target object, a preset threshold is determined, and based on the preset threshold, the motion state of the target object is analyzed to obtain an analysis result, where the analysis result includes: whether the target object is in a stationary state and the stationary duration when the target object is in a stationary state, and the preset threshold is used to analyze whether the target object is in a stationary state; Based on the analysis result, it is judged whether the target object meets the second warning condition to obtain a second judgment result, and the first judgment result and the second judgment result are determined as the target judgment result.

5. The warning method based on regional video according to claim 4, wherein When the target judgment result indicates that the target object meets the warning condition of the target area, a target warning message is generated. It further includes: When the target judgment result indicates that the target object meets the first warning condition, based on the color of the target object and the target type of the target object, a first warning message is determined; When the target judgment result indicates that the target object meets the second warning condition, based on the analysis result and the first warning message, the target warning message is determined; When the target judgment result indicates that the target object does not meet the second warning condition, the first warning message is determined as the target warning message.

6. The warning method based on regional video according to claim 1, wherein, After inputting the image frame in the video data into the target deep learning model and outputting the recognition result of the target object in the image frame, it further includes: Determine the contour of the target object in the image frame to obtain a target contour; In the image frame, extract the image of the inscribed rectangle part of the target contour to obtain a target image; Segment the target image and filter out the images in the preset area according to the preset filtering strategy to obtain M image blocks, where the preset area includes: the area that interferes with the color of the target object, and M is a positive integer; Based on the M image blocks, determine the color of the target object.

7. The warning method based on regional video according to claim 6, wherein Based on the M image blocks, determining the color of the target object includes: Determine the average value of the color values of the RGB channels of the M image blocks to obtain a target color average value; Obtain the color values of the RGB channels of the standard color to obtain a standard color value; Calculate the difference between the target color average value and the standard color value to obtain a calculation result; Based on the calculation result, determine the color of the target object.

8. The warning method based on regional video according to claim 4, wherein, Based on the pixels of the target object, determining the preset threshold includes: Determine the pixel width and pixel height of the target object in the image frame; Calculate the product of the pixel width and the first preset value to obtain a first value; Calculate the product of the pixel height and the second preset value to obtain a second value; Compare the first value and the second value to obtain a comparison result, and based on the comparison result, determine the preset threshold.

9. An alarm device based on regional video, characterized in that, It includes: An acquisition unit for acquiring video data of the target area; A first processing unit, configured to input an image frame in the video data into a target deep learning model, and output an identification result of a target object in the image frame, wherein the identification result of the target object includes: N types of object types and feature information of the target object, the target deep learning model includes: a deep learning model trained based on a pixel size and a probability distribution model, the probability distribution model is used to represent the distribution law of data, and the N types of object types include at least one of the following: item type, species type, and N is an integer; A determination unit, configured to determine a target type of the target object based on the identification result of the target object, wherein the target type is one of the N types of object types; A second processing unit, configured to determine whether the target object meets an alarm condition of the target area based on the target type of the target object, obtain a target determination result, and generate a target alarm information when the target determination result indicates that the target object meets the alarm condition of the target area.

10. A computer program product comprising computer instructions, characterized in that, When the computer instruction is executed by a processor, the steps of the alarm method based on regional video according to any one of claims 1 to 8 are implemented.

Images