Detection model construction method, detection method, device, server and medium

By building a conditional generation adversarial network model and classification network model, using video information preprocessing to obtain throwing area information, solving the problems of low efficiency and high cost of violent throwing behavior detection in the existing technology, and achieving automatic detection and accuracy improvement .

CN111985269BActive Publication Date: 2024-12-03SF TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910426005.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-05-21
Publication Date
2024-12-03
Estimated Expiration
2039-05-21

AI Technical Summary

Technical Problem

In the prior art, the investigation of violent throwing and throwing behavior through artificial visual viewing of video information collected by cameras has problems such as strong subjectivity, low detection efficiency and high cost.

Method used

Confirmation of adversarial network models and classification network models are constructed, and throwing area information is obtained through video information preprocessing, and classification models are trained to automatically detect violent throwing behavior, improving detection efficiency and accuracy.

Benefits of technology

Automatic detection of violent throwing and throwing behavior is realized, the detection efficiency and accuracy are improved, the detection cost is reduced, and the calculation amount and training speed of the model are simplified.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111985269B_ABST
    Figure CN111985269B_ABST
Patent Text Reader

Abstract

The present application discloses a detection model construction method, a detection method, a device, a server and a medium, the method comprising: obtaining video information of a package operation behavior; preprocessing the video information to obtain a first image frame sequence; training a conditional generative adversarial network model based on the first image frame sequence; processing the first image frame sequence based on the throwing area information to obtain a second image frame sequence; training a classification network model based on the second image frame sequence. The embodiment of the present application can determine the throwing area information of the image frame of the package operation behavior by constructing a conditional generative adversarial network model, so that the throwing area information can be used to construct a classification model, reducing the amount of calculation of the model, and the constructed classification model can be used to realize automatic detection of the operator's violent throwing behavior, thereby improving the detection efficiency and accuracy of violent throwing behavior and reducing the detection cost.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application generally relates to the field of image processing technology, and specifically relates to a detection model construction method, a detection method, a device and a medium. Background Art

[0002] With the development of the logistics industry, the number of parcels generated every day is increasing. During the parcel handling and sorting operations of couriers, violent throwing often occurs, which affects the image of the courier company, causes damage to the parcels, and ultimately causes economic losses to the courier company.

[0003] At present, in order to supervise the operating standards of couriers and investigate violent throwing phenomena, the couriers' operations are captured through cameras, and artificial vision is used to view the video information collected by the cameras to investigate violent throwing behaviors.

[0004] Manual investigation of violent throwing behavior is highly subjective, has low detection efficiency and high cost. Summary of the invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a detection model construction method, a detection method and a device to improve the efficiency of violent throwing detection.

[0006] In a first aspect, an embodiment of the present application provides a method for constructing a violent throwing detection model, the method comprising:

[0007] Get video information of package operation behavior;

[0008] Preprocessing the video information to obtain a first image frame sequence;

[0009] Based on the first image frame sequence, training and generating a conditional generative adversarial network model, the conditional generative adversarial network model outputs throwing area information corresponding to the first image frame sequence;

[0010] Processing the first image frame sequence based on the throwing area information to obtain a second image frame sequence;

[0011] Based on the second image frame sequence, a classification network model is trained and generated, wherein the classification network model is used to detect whether the package operation behavior to be detected is a violent throwing behavior.

[0012] In a second aspect, an embodiment of the present application provides a detection method, including: obtaining video information of a package operation behavior to be detected;

[0013] Preprocessing the video information to obtain a third image frame sequence;

[0014] Inputting the third image frame sequence into the conditional generative adversarial network model described in the first aspect, and outputting the throwing area information corresponding to the third image frame sequence;

[0015] Processing the third image frame sequence based on the throwing area information to obtain a fourth image frame sequence;

[0016] The fourth image frame sequence is input into the classification network model as described in the first aspect, and a label of the package operation behavior to be detected is output, wherein the label indicates whether the package operation behavior to be detected is a violent throwing behavior.

[0017] In a third aspect, an embodiment of the present application provides a detection model building device, including:

[0018] An acquisition module is used to obtain video information of package operation behavior;

[0019] A first processing module, used for preprocessing the video information to obtain a first image frame sequence;

[0020] A first generating module, configured to train and generate a conditional generative adversarial network model based on the first image frame sequence, wherein the conditional generative adversarial network model outputs throwing area information corresponding to the first image frame sequence;

[0021] A second processing module, configured to process the first image frame sequence based on the throwing area information to obtain a second image frame sequence;

[0022] The second generating module is used to train and generate a classification network model based on the second image frame sequence, and the classification network model is used to detect whether the package operation behavior to be detected is a violent throwing behavior.

[0023] In a fourth aspect, an embodiment of the present application provides a detection device, including:

[0024] An acquisition module, used to acquire video information of the operation package behavior to be detected;

[0025] A first processing module, used for preprocessing the video information to obtain a third image frame sequence;

[0026] A determination module, configured to input the third image frame sequence into the conditional generative adversarial network model as described in the first aspect, and output the throwing area information corresponding to the third image frame sequence;

[0027] A second processing module, configured to process the third image frame sequence based on the throwing area information to obtain a fourth image frame sequence;

[0028] A detection module is used to input the fourth image frame sequence into the classification network model as described in the first aspect, and output a label of the operating package behavior to be detected, wherein the label indicates whether the operating package behavior to be detected is a violent throwing behavior.

[0029] In a fifth aspect, an embodiment of the present application provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect or the second aspect when executing the program.

[0030] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to implement the method as described in the first aspect or the second aspect.

[0031] In summary, the detection model construction method, detection method, device, server and medium provided in the embodiments of the present application can determine and obtain the throwing area information of the image frame of the package operation behavior by constructing a conditional generation adversarial game network model, so that the throwing area information can be used to construct a classification model, reduce the calculation amount of the model, and improve the training speed of the classification model. The constructed classification model can then be used to realize automatic detection of the operator's violent throwing behavior, thereby improving the detection efficiency and accuracy of violent throwing behavior and reducing the detection cost.

[0032] Furthermore, by preprocessing the segmented frame set through the background algorithm, the focus can be placed on the motion area, thereby reducing the difficulty of learning the conditional generative adversarial network model and making it easier to converge.

[0033] Furthermore, the throwing area information determined by the conditional generative adversarial network model is used to generate the original image sequence and optical flow sequence to train the generative classification network model, thereby improving the discrimination accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0035] Figure 1 A schematic diagram of a process for constructing a violent throwing detection model according to an embodiment of the present application;

[0036] Figure 2 A flowchart of a method for constructing a violent throwing detection model according to another embodiment of the present application;

[0037] Figure 3 A schematic diagram of a violent throwing detection method according to an embodiment of the present application;

[0038] Figure 4A schematic diagram of the structure of a violent throwing detection model building device according to an embodiment of the present application;

[0039] Figure 5 It is a structural schematic diagram of a violent throwing detection device according to an embodiment of the present application;

[0040] Figure 6 A schematic diagram of the structure of a computer system of a server according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant application, rather than to limit the application. It is also necessary to explain that, for ease of description, only the parts related to the application are shown in the accompanying drawings.

[0042] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0043] It can be understood that the violent throwing detection model involved in the embodiment of the present application can be used in actual logistics to detect the courier's operating behavior on the package, so as to determine whether the courier's behavior has involved violent throwing, so as to provide supervision and management of the couriers by managers.

[0044] For ease of understanding and explanation, the following Figure 1 to Figure 2 The construction of the violent throwing detection model provided in the embodiment of the present application is described. As shown in Figure 1, a flow chart of the method for constructing the violent throwing detection model provided in the embodiment of the present application is shown in Figure 1. Figure 1 As shown, the method includes:

[0045] S110, obtaining video information of package operation behavior.

[0046] S120: pre-process the video information to obtain a first image frame sequence.

[0047] S130: Based on the first image frame sequence, training a conditional generative adversarial network model.

[0048] S140: Process the first image frame sequence based on the throwing area information to obtain a second image frame sequence.

[0049] S150: Based on the second image frame sequence, training and generating a classification network model.

[0050] Specifically, the violent throwing detection model construction method provided by the implementation of the present application can first obtain video information of the courier operating the package in an online or offline manner, and then pre-process the obtained video information to obtain an image frame sequence, such as a plurality of short videos including 10 frames. Then, the image frame sequence can be used to train a conditional generative adversarial network model (Conditional Generative Adversarial Nets, CGAN) so that the CGAN can output the throwing area information of the image frame sequence. Then, the throwing area information of the image frame sequence determined by the CGAN can be used to process the first image frame sequence to intercept the area corresponding to the throwing area information and generate a second image frame sequence. Finally, the generated second image frame sequence can be used to train the classification network model to obtain a classification network model with a perfect mapping algorithm. The classification network model can be used to analyze the video information of the operation package behavior to be detected to determine whether there is a violent throwing behavior in the video information.

[0051] It can be understood that the training sample of the classification network model can be the first image frame sequence initially obtained for training CGAN, that is, when the CGAN is constructed, the output result of the CGAN corresponding to the first image frame sequence is used to train the classification network model. Alternatively, the video information of the operation package behavior can be re-acquired, the image frame sequence is obtained through pre-processing, and then the throwing area information of the image in the re-acquired video information is determined by CGAN to train and generate the classification network model, which is not limited in the embodiment of the present application.

[0052] The violent throwing detection model construction method provided in the embodiment of the present application can determine and obtain the throwing area information of the image frame of the operation package behavior by constructing a conditional generation adversarial game network model, so that the throwing area information can be used to construct a classification model, reduce the calculation amount of the model, and improve the training speed of the classification model. The constructed classification model can then be used to realize automatic detection of the operator's violent throwing behavior, thereby improving the detection efficiency and accuracy of violent throwing behavior and reducing the detection cost.

[0053] In order to better explain the method of this application, Figure 2 Explain in detail the process of building a violent throwing detection model. Figure 2 As shown, the method includes:

[0054] S210, obtaining video information of package operation behavior.

[0055] S221, segment the video information to generate at least one frame set.

[0056] S222: Process the pictures in the frame set based on the background algorithm to generate a first image frame sequence.

[0057] Specifically, the operation of the courier on the package can be filmed by a camera to generate a video stream, and then the captured video stream is imaged to generate a frame set. For example, the obtained video is divided into multiple short videos, such as 10 frames as a frame set.

[0058] When obtaining a certain segment of 10-frame image set, first extract the first frame image, then extract the next frame image, and judge whether the similarity (SSIM) between the two adjacent frames is less than a threshold value, such as 0.95. If it is less than, it means that the package has shifted, retain the frame image, continue to extract the next frame, and judge. If it is not less than, it means that the package has not shifted, discard the frame image, and continue to extract the next frame until a set of 10 frames is obtained.

[0059] Furthermore, after obtaining the image frame sequence of the short video, the pictures in the frame set can be input into a background modeling algorithm, such as a mixed Gaussian background modeling algorithm, a mog background modeling algorithm or a gmg background modeling algorithm, to preprocess the image frames to output image frames that only include motion feature data, that is, to generate a first image frame sequence, such as a motion map.

[0060] It can be understood that after the image frame is processed by the background algorithm, the feature data of the image can be focused on the motion area, thereby reducing the difficulty of network learning and making the network model easier to converge.

[0061] It can also be understood that after the image frame sequence is processed by the background algorithm, the RGB sequence or the optical flow sequence can be further extracted as sample data for training CGAN. For example, the optical flow algorithm is used to pre-process the image set to generate the optical flow sequence.

[0062] S230, training a conditional generative adversarial network model based on the first image frame sequence.

[0063] Specifically, after obtaining the first image frame sequence as sample data after preprocessing, the obtained image frame sequence can be input into the CGAN algorithm to train the CGAN network model, that is, continuously adjust the parameters of the CGAN network model to construct the optimal CGAN network model, and make the constructed network model output the throwing area information of the image frame sequence. Among them, the first image frame sequence input by the CGAN network model is stacked into a corresponding number of channels, and the input format of the image frame can be 1*10*h*w, where h and w are scale information. The throwing area information output by the CGAN network model can be the throwing area coordinate value, and can be displayed as a darkened image. If the output is an irregular arbitrary figure, the coordinate value can represent the coordinate value of the endpoint of the circumscribed rectangle of the irregular figure.

[0064] It can be understood that as a supervised machine learning, it is necessary to first generate standard label samples manually, that is, manually annotate the throwing area information of the image frame. For example, the generated label sample is a throwing area (mask) map with the same size as the above image frame, the pixel value of the thrown area can be 255, and the rest can be 0, and the input format is h*w.

[0065] The CGAN network model includes a generator and a discriminator. During the training process, the parameters of the generator and the discriminator are gradually adjusted by calculating the loss function value of the CGAN network model, and the training operation is ended by judging the loss function value or the number of iterations. It can be understood that the goal of training the CGAN model is to generate a mask image that is infinitely close to the mask image of the input label sample.

[0066] S241, determining a region of interest sequence of images in the first image frame sequence based on the discarding region information.

[0067] S242, normalizing the region of interest based on the scale information so that the normalized image frame meets the input of the classification network model.

[0068] S243: Generate a second image frame sequence based on the region of interest sequence.

[0069] Specifically, after training and generating CGAN and obtaining the output result of CGAN corresponding to the first image frame sequence, the first image frame sequence and its corresponding CGAN output result can be used to continue training the classification network model. That is, after obtaining the discarding area information of the CGAN output corresponding to the first image frame sequence, the image of the corresponding first image frame sequence can be cut out according to the discarding area information to obtain the region of interest (ROI Region Of Interest, ROI). In detail, the discarding area of ​​the image output by CGAN can be matched to the input image, the area can be deducted, and the rest can be discarded, so that only the feature data in the area is processed in subsequent calculations.

[0070] For example, after obtaining the mask image of the output result of CGAN, the bbox corresponding to the mask image can be further obtained, such as a regular circumscribed rectangle. Then, based on the bbox, the ROI region sequence image corresponding to the image frame can be found. If the coordinates of the upper left corner and the lower right corner of the generated bbox are (100,100) and (300,300) respectively, the ROI sequence is to intercept the 200*200 area in the range of (100,100) to (300,300) in each image frame as the ROI sequence image.

[0071] Furthermore, since the input image scale information of the classification model needs to be consistent, such as 300*300, it is necessary to normalize the obtained ROI area so that the image required for the corresponding classification algorithm can be obtained. That is, 50 rows and columns of pixel values ​​are added to the top, bottom, left, and right of the 200*200 area to expand it to 300*300. The pixel value size can be 128 (grayscale image) or (104, 117, 123), which are the average values ​​of the three BGR channels respectively.

[0072] For example, through the above normalization processing, the generated bbox image can be no larger than 300*300 according to the acquired scale information. If the scale information is 2, the bounding rectangle of the mask image generated by GGAN is a bbox with coordinates of the upper left corner and the lower right corner (100,100)(300,300), and the scale information of this area is 2, then the bbox needs to be resized to 400*400.

[0073] It can be understood that in order to improve the discrimination accuracy of the classification network model, a classification network model of TSN (Temporal Segment Networks) can be constructed, that is, based on two-stream. Therefore, after obtaining the input image frame that meets the classification network model, such as an RGB image, it is also necessary to calculate the optical flow sequence based on the generated RGB frame, such as using the TVL1 algorithm, so as to generate a second image frame sequence including the original image sequence and the optical flow sequence.

[0074] It can be understood that the above S242 can be selectively executed, that is, when the scale of the determined region of interest meets the input of the classification network model, it is not necessary to perform normalization processing on it, and S243 can be directly executed.

[0075] S250: Based on the second image frame sequence, training and generating a classification network model.

[0076] Specifically, when the second image frame sequence, i.e., the training sample data of the classification network model, is obtained, the second image sequence can be trained based on the classification algorithm to adjust the parameters of the algorithm, and finally obtain a trained mapping relationship, i.e., the classification network model. The output of the classification network model can be different behavior labels, i.e., whether the behavior corresponding to the image frame sequence is violent throwing.

[0077] For example, in practice, three output results may be defined, including no throwing, slight throwing, and severe throwing. Behavior labels such as output 1 and 0 binary characters may be defined to represent different behavior categories.

[0078] The violent throwing detection model construction method provided in the embodiment of the present application can use the video information to train and generate a CGAN network model after obtaining the video information of the package operator's behavior, so as to use the CGAN network model to search for the throwing area corresponding to the obtained image frame sequence to obtain the throwing area, so as to generate a classification model by training the obtained throwing area. Compared with analyzing and training the original image, this reduces the complexity of learning and improves the learning efficiency.

[0079] It can be understood that after constructing the violent throwing detection model through the above method, the model can be used to judge the actual behavior of couriers to confirm whether violent throwing occurs. The specific detection method is as follows: Figure 3 shown.

[0080] Figure 3 The figure is a flow chart of a violent throwing detection method provided in an embodiment of the present application. As shown in the figure, the method includes:

[0081] S310, obtaining video information of the package operation behavior to be detected.

[0082] S320: pre-process the video information to obtain a third image frame sequence.

[0083] S330, input the third image frame sequence into the CGAN model.

[0084] S340: Process the third image frame sequence according to the throwing area information to obtain a fourth image frame sequence.

[0085] S350: Input the fourth image frame sequence into the classification network model, and output a label of the operation package behavior to be detected.

[0086] Specifically, in actual detection, the video information of the current operator behavior can be obtained online, or the previous video information can be obtained by offline download. For example, the input time of video information can be set according to the working hours of the management personnel, and the video information of the whole day can be downloaded for analysis before leaving get off work every day, or the video information of the previous three days can be downloaded each time to input into the detection model to monitor the courier's operation behavior.

[0087] Similarly, before inputting into the pre-established CGAN model, the acquired video information still needs to be pre-processed to obtain an image frame sequence. For example, the image frame sequence can be processed using a background algorithm to remove redundant data features. The obtained image frame sequence can then be input into the CGAN model to output the corresponding throwing area information.

[0088] After obtaining the throwing area information corresponding to the image frame, the frame sequence obtained in the previous step can be further processed according to the output throwing area information to obtain an image frame that only includes the throwing area, and an optical flow sequence is generated according to the image frame, so that the image frame and the optical flow sequence are used as the input of the classification model (i.e., the fourth frame sequence). Finally, the fourth frame sequence can be input into the constructed classification model, and the behavior label corresponding to the frame sequence can be output. For example, 0 and 1, or other characters are output to indicate whether the behavior is violent throwing, or whether it is severe violent throwing.

[0089] It can be understood that the generation process of the third frame sequence is similar to the generation process of the first frame sequence mentioned above, and the generation process of the fourth image frame sequence is similar to the generation process of the second image frame sequence mentioned above, and also includes the frame sequence and optical flow sequence of the original image, which will not be repeated here.

[0090] Furthermore, during the violent throwing detection process, the distance at which the package was thrown can be calculated based on the output results and scale information of the CGAN model to finally determine whether violent throwing behavior occurred or the degree of violent throwing.

[0091] It is understood that, in practice, the calculation can be performed in conjunction with the output of the classification model. For example, if the output of the classification model indicates that the current behavior is violent throwing, the throwing distance can be further calculated to verify whether violent throwing actually occurs, or to confirm the degree of violent throwing. Alternatively, if the output of the classification model indicates that the current behavior is not violent throwing, it can still be further verified by the throwing distance. This application does not limit this.

[0092] The violent throwing detection method provided in the embodiment of the present application can first segment the video information after acquiring the video information of the operator's behavior, and then process the segmented frame sequence using the background algorithm to obtain a first frame sequence, so that the feature data is focused on the motion area. When it is input into the CGAN model, the difficulty of learning the CGAN model can be reduced, making it easier for the CGAN model to converge to obtain the throwing area information corresponding to each image in the frame sequence, and then the first frame sequence can be processed according to the throwing area information, and the feature data is further focused on the throwing area, so as to obtain a second frame sequence including the original image frame and the optical flow sequence, so that the classification model can efficiently output the behavior label corresponding to the image frame sequence according to the second frame sequence, thereby improving the efficiency and accuracy of violent throwing detection.

[0093] Figure 4 FIG. 1 is a schematic diagram of the structure of a violent throwing detection model building device provided in an embodiment of the present application. Figure 4 As shown, the apparatus 400 may include:

[0094] An acquisition module 410 is used to acquire video information of package operation behavior;

[0095] A first processing module 420 is used to pre-process the video information to obtain a first image frame sequence;

[0096] A first generating module 430 is used to train and generate a conditional generative adversarial network model based on the first image frame sequence, and the conditional generative adversarial network model outputs the throwing area information corresponding to the first image frame sequence;

[0097] A second processing module 440 is used to process the first image frame sequence based on the throwing area information to obtain a second image frame sequence;

[0098] The second generating module 450 is used to train and generate a classification network model based on the second image frame sequence, and the classification network model is used to detect whether the package operation behavior to be detected is a violent throwing behavior.

[0099] Preferably, in the violent throwing detection model building device provided in the embodiment of the present application, the first processing module 420 includes:

[0100] The segmentation unit 421 is used to segment the video information to generate at least one frame set;

[0101] The processing unit 422 processes the frame set based on the background algorithm to generate a first image frame sequence.

[0102] Preferably, in the violent throwing detection model building device provided in the embodiment of the present application, the second processing module 440 includes:

[0103] A determining unit 441 is used to determine a region of interest sequence of images in the first image frame sequence based on the throwing region information;

[0104] The generating unit 442 is configured to generate a second image frame sequence based on the region of interest sequence, where the second image frame sequence includes an original image sequence and an optical flow sequence.

[0105] Figure 5 FIG. 1 is a schematic diagram of the structure of a violent throwing detection device provided in an embodiment of the present application. Figure 5 As shown, the apparatus 500 may include:

[0106] An acquisition module 510 is used to acquire video information of the operation package behavior to be detected;

[0107] A first processing module 520, configured to pre-process the video information to obtain a third image frame sequence;

[0108] A determination module 530 is used to input the third image frame sequence into the conditional generative adversarial network model as described in the above embodiment, and output the throwing area information corresponding to the third image frame sequence;

[0109] A second processing module 540 is used to process the third image frame sequence based on the throwing area information to obtain a fourth image frame sequence;

[0110] The detection module 550 is used to input the fourth image frame sequence into the classification network model as described in the above embodiment, and output a label of the package operation behavior to be detected, where the label indicates whether the package operation behavior to be detected is a violent throwing behavior.

[0111] On the other hand, an embodiment of the present application also provides a server, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the violent throwing detection method as described above is implemented.

[0112] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 500 suitable for implementing a server of an embodiment of the present application.

[0113] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 603 to a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0114] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 606 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0115] In particular, according to the embodiment of violent throwing detection disclosed in this application, the above reference Figure 1The described process may be implemented as a computer software program. For example, the violent throw detection embodiments disclosed in the present application include a computer program product, which includes a computer program tangibly embodied on a machine-readable medium, the computer program including a computer program for executing Figure 1 In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 609 and / or installed from the removable medium 611.

[0116] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or device.

[0117] In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0118] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various violent throwing detection embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the aforementioned module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] The units or modules involved in the embodiments of the present application may be implemented in software or in hardware. The units or modules described may also be arranged in a processor. For example, they may be described as: a processor includes an acquisition module, a first processing module, a first generation module, a second processing module, and a second generation module. Among them, the names of these units or modules do not constitute a limitation on the units or modules themselves under certain circumstances. For example, the first calculation module may also be described as "used to train and generate a classification network model based on the second image frame sequence, and the classification network model is used to detect whether the operation of the package to be detected is a violent throwing behavior."

[0120] As another aspect, the present application further provides a computer-readable storage medium, which may be the computer-readable storage medium included in the aforementioned device in the above embodiment; or may be a computer-readable storage medium that exists independently and is not assembled into the device. The computer-readable storage medium stores one or more programs, and the aforementioned programs are used by one or more processors to execute the method for determining violent throwing detection described in the present application, specifically executing:

[0121] Get video information of package operation behavior;

[0122] Preprocessing the video information to obtain a first image frame sequence;

[0123] Based on the first image frame sequence, training and generating a conditional generative adversarial network model, the conditional generative adversarial network model outputs throwing area information corresponding to the first image frame sequence;

[0124] Processing the first image frame sequence based on the throwing area information to obtain a second image frame sequence;

[0125] Based on the second image frame sequence, a classification network model is trained and generated, where the classification network model is used to detect whether the package operation to be detected is a violent throwing behavior.

[0126] In summary, the detection model construction method, detection method, device, server and medium provided in the embodiments of the present application can determine and obtain the throwing area information of the image frame of the package operation behavior by constructing a conditional generation adversarial game network model, so that the throwing area information can be used to construct a classification model, reduce the calculation amount of the model, and improve the training speed of the classification model. The constructed classification model can then be used to realize automatic detection of the operator's violent throwing behavior, thereby improving the detection efficiency and accuracy of violent throwing behavior and reducing the detection cost.

[0127] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of application involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the concept of the application. For example, the above features are replaced with the technical features with similar functions disclosed in the present application (but not limited to) by each other to form a technical solution.

Claims

1. A detection model construction method, characterized in that: The method comprises: Get video information of package operation behavior; Extracting images in the video information frame by frame, and determining whether the similarity between two adjacent frames of images is less than a threshold; if the similarity is less than the threshold, retaining the current frame of image, otherwise, discarding the current frame of image, and obtaining a preset number of image sets; Preprocessing the image set to obtain a first image frame sequence; Based on the first image frame sequence, training and generating a conditional generative adversarial network model, the conditional generative adversarial network model outputs throwing area information corresponding to the first image frame sequence; Performing a matting process on the first image frame sequence according to the discarding area information to obtain an area of ​​interest sequence; calculating an optical flow sequence based on the area of ​​interest sequence, and generating a second image frame sequence according to the area of ​​interest sequence and the optical flow sequence; Based on the second image frame sequence, a classification network model is trained and generated, where the classification network model is used to detect whether the package operation to be detected is a violent throwing behavior.

2. The detection model construction method according to claim 1, characterized in that: The preprocessing of the image set to obtain a first image frame sequence comprises: Segmenting the image set to generate at least one frame set; The pictures in the frame set are processed based on a background algorithm to generate the first image frame sequence.

3. The detection model construction method according to claim 1, characterized in that: Before generating a second image frame sequence based on the region of interest, the method further includes: The region of interest is normalized based on scale information so that the normalized image frame meets the input of the classification network model.

4. A detection method, characterized in that: The method comprises: Obtain video information of the package operation behavior to be detected; Preprocessing the video information to obtain a third image frame sequence; Inputting the third image frame sequence into the conditional generative adversarial network model according to any one of claims 1 to 3, and outputting the throwing area information corresponding to the third image frame sequence; Processing the third image frame sequence based on the throwing area information to obtain a fourth image frame sequence; The fourth image frame sequence is input into the classification network model as described in any one of claims 1 to 3, and a label of the package operation behavior to be detected is output, wherein the label indicates whether the package operation behavior to be detected is a violent throwing behavior.

5. A detection model building device, characterized in that: The device comprises: An acquisition module is used to obtain video information of package operation behavior; A first processing module is used to extract images from the video information frame by frame, and determine whether the similarity between two adjacent frames of images is less than a threshold; if the similarity is less than the threshold, retain the current frame of image, otherwise, discard the current frame of image, and obtain a preset number of image sets; pre-process the image set to obtain a first image frame sequence; A first generating module, configured to train and generate a conditional generative adversarial network model based on the first image frame sequence, wherein the conditional generative adversarial network model outputs throwing area information corresponding to the first image frame sequence; A second processing module is used to perform a matting process on the first image frame sequence according to the discarding area information to obtain an area of ​​interest sequence; calculate an optical flow sequence based on the area of ​​interest sequence, and generate a second image frame sequence according to the area of ​​interest sequence and the optical flow sequence; The second generating module is used to train and generate a classification network model based on the second image frame sequence, and the classification network model is used to detect whether the package operation behavior to be detected is a violent throwing behavior.

6. The detection model building device according to claim 5, characterized in that: The first processing module comprises: A segmentation unit, configured to segment the image set to generate at least one frame set; The processing unit processes the frame set based on a background algorithm to generate a first image frame sequence.

7. A detection device, characterized in that: The device comprises: An acquisition module, used to acquire video information of the operation package behavior to be detected; A first processing module, used for preprocessing the video information to obtain a third image frame sequence; A determination module, configured to input the third image frame sequence into the conditional generative adversarial network model according to any one of claims 1 to 3, and output the throwing area information corresponding to the third image frame sequence; A second processing module, configured to process the third image frame sequence based on the throwing area information to obtain a fourth image frame sequence; A detection module is used to input the fourth image frame sequence into the classification network model as described in any one of claims 1 to 3, and output a label of the operating package behavior to be detected, wherein the label indicates whether the operating package behavior to be detected is a violent throwing behavior.

8. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 3 or claim 4 is implemented.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is used to implement the method according to any one of claims 1 to 3 or claim 4.