Event behavior mode speculation method and device based on image recognition and storage medium
By collecting and extracting images of the event site, combining evidence identification models, and inferring event behavior patterns, the problem of lack of automated assistance in event restoration is solved, and efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202410165617.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-05
- Publication Date
- 2025-05-30
AI Technical Summary
During the event restoration process, relevant personnel need to actively speculate on the behavior of the event, lack of automated assistance, and have low efficiency and accuracy.
By collecting images at the event site, performing feature extraction and edge detection, inputting preset evidence identification models, obtaining evidence categories and feature information, and inferring event behavior patterns based on the basic information of the event.
Automatic inference of event behavior is realized, the efficiency and accuracy of the event restoration process is improved, and the speculation burden of relevant personnel is reduced.
Smart Images
Figure CN120069053A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image recognition technology, and in particular, to a method, device, and storage medium for inferring event behavior patterns based on image recognition. Background Art
[0002] When restoring an event, relevant personnel usually need to infer the event behavior pattern based on the on-site investigation results. Currently, most of the inferences are made actively by relevant personnel. Therefore, there is an urgent need for a method that can automatically infer the event behavior pattern to assist relevant personnel in restoring the event occurrence process. Summary of the Invention
[0003] The purpose of the embodiments of this application is to provide a method, device, and storage medium for inferring event behavior patterns based on image recognition to solve the above technical problems.
[0004] On the one hand, a method for inferring event behavior patterns based on image recognition is provided. The method includes:
[0005] Collect images of the event site to obtain event site images;
[0006] Perform feature extraction processing on the event site images to obtain image feature data of the event site images;
[0007] Input the image feature data into a preset evidence recognition model to obtain evidence category information of the evidence in the event site images;
[0008] Obtain evidence feature information of the evidence in the event site images;
[0009] Obtain basic event information of the event associated with the event site images; the basic event information includes type information of the event and scene information of the scene where the event occurs;
[0010] Infer the event behavior pattern of the event site based on the evidence category information, the evidence feature information, and the basic event information.
[0011] In one of the embodiments, the performing feature extraction processing on the event site images to obtain image feature data of the event site images includes:
[0012] Perform bilateral filtering processing on the event site images to obtain filtered images after filtering processing;
[0013] Perform edge detection processing on the filtered images to obtain edge detection images;
[0014] Extract first image feature data of each pixel point in the edge detection images;
[0015] Use the first image feature data as the image feature data of the event scene image.
[0016] In one embodiment, the edge detection process on the filtered image to obtain an edge detection image includes:
[0017] When it is determined that edge detection processing needs to be performed on the filtered image, perform edge detection processing on the filtered image to obtain an edge detection image.
[0018] In one embodiment, the determination of the need to perform edge detection processing on the filtered image includes:
[0019] Start timing when the filtered image is obtained. If an edge detection instruction is received within a preset time period, it is determined that edge detection processing needs to be performed on the filtered image;
[0020] And / or,
[0021] Perform grayscale processing on the event scene image and the filtered image respectively to obtain an event scene grayscale image and a filtered grayscale image. When the coincidence degree between the event scene grayscale image and the filtered grayscale image is less than a preset coincidence degree threshold, it is determined that edge detection processing needs to be performed on the filtered image.
[0022] In one embodiment, the bilateral filtering process on the event scene image to obtain a filtered image after filtering includes:
[0023] Determine the image type of the event scene image;
[0024] Determine the spatial information weighting value and the pixel information weighting value corresponding to the image type according to a preset correspondence table between image type and weighting information; the preset correspondence table between image type and weighting information includes the correspondence between image type, spatial information weighting value, and pixel information weighting value;
[0025] Perform bilateral filtering on the event scene image based on the spatial information weighting value and the pixel information weighting value to obtain a filtered image after filtering.
[0026] In one embodiment, before inputting the image feature data into a preset evidence recognition model, the method includes:
[0027] Obtain a historical event scene image set;
[0028] Perform bilateral filtering and edge detection processing on each historical event scene image in the historical event scene image set in sequence to obtain a historical edge detection image;
[0029] Label the categories of evidence in each of the historical edge detection images to obtain sample images, and use the set of all the sample images as the training sample image set;
[0030] Train an evidence recognition model based on the training sample image set and a preset training model.
[0031] In one embodiment, after inputting the image feature data into a preset evidence recognition model to obtain the evidence category information of the evidence in the event scene image, the method includes:
[0032] Label the evidence in the event scene image according to the evidence category information;
[0033] Add the event scene image labeled with the evidence category information to the training sample image set to obtain a new training sample image set;
[0034] Perform model training based on the new training sample image set to obtain a new evidence recognition model;
[0035] When obtaining the image feature data corresponding to a new event scene image, input the image feature data corresponding to the new event scene image into the new evidence recognition model to obtain the evidence category information of the evidence in the new event scene image.
[0036] In one embodiment, the inferring the event behavior mode of the event scene based on the evidence category information, the evidence feature information, and the basic event information includes:
[0037] Match in a preset database the target evidence event record with the highest comprehensive matching degree for the evidence type information, the evidence feature information, and the basic event information; the database stores multiple evidence event records, and each evidence event record includes historical evidence type information, historical evidence feature information, historical basic event information, and the corresponding historical event behavior mode;
[0038] Infer the event behavior mode of the event scene based on the historical event behavior mode in the target evidence event record.
[0039] On the other hand, an electronic device is provided, including a processor and a memory. A computer program is stored in the memory, and the processor executes the computer program to implement any one of the above methods.
[0040] On the other hand, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, any one of the above methods is implemented.
[0041] The method, device, and storage medium for inferring event behavior patterns based on image recognition provided by this application obtain an event scene image by collecting images of the event scene, perform feature extraction processing on the event scene image to obtain image feature data of the event scene image, input the image feature data into a preset evidence recognition model to obtain evidence category information of the evidence in the event scene image, obtain evidence feature information of the evidence in the event scene image, obtain basic event information of the event associated with the event scene image, infer the event behavior pattern of the event scene based on the evidence category information, evidence feature information, and basic event information. It realizes the automatic recognition of evidence based on image recognition technology, and infers the event behavior pattern of the event scene based on the recognized evidence, evidence type information, and basic event information, realizing the function of automatic identification and analysis of event behavior patterns, and can assist relevant personnel in restoring the event occurrence process. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is a schematic flowchart of the method for inferring event behavior patterns based on image recognition provided in Embodiment 1 of this application;
[0043] Figure 2 It is a schematic flowchart of the feature extraction processing performed on the event scene image provided in Embodiment 1 of this application;
[0044] Figure 3 It is a schematic structural diagram of the electronic device provided in Embodiment 2 of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0045] In order to make the objectives, technical solutions, and advantages of this application clearer, the following further elaborates on this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain this application and are not used to limit this application.
[0046] Embodiment 1:
[0047] In traditional event investigation scenarios, evidence is identified manually, and the event behavior pattern of the event scene is inferred manually. It relies on the experience of relevant personnel, the process is not standardized, and the investigation efficiency is relatively low. In response to this, Embodiment 1 of this application provides a method for inferring event behavior patterns based on image recognition. Please refer to Figure 1 as shown, and it includes the following steps:
[0048] S11: Collect images of the event scene to obtain an event scene image.
[0049] S12: Perform feature extraction processing on the event scene image to obtain image feature data of the event scene image.
[0050] S13: Input the image feature data into a preset evidence recognition model to obtain the evidence category information of the evidence in the event scene image.
[0051] S14: Obtain the evidence feature information of the evidence in the event scene image.
[0052] S15: Obtain the basic event information of the event associated with the event scene image; the basic event information includes the type information of the event and the scene information of the scene where the event occurs.
[0053] It should be noted that in addition to the type information of the event and the scene information of the scene where the event occurs, the basic event information may also include the event occurrence time, event occurrence location, etc.
[0054] S16: Speculate the event behavior mode at the event scene based on the evidence category information, evidence feature information, and basic event information.
[0055] Next, the specific processes of the above steps will be described in detail.
[0056] The method for speculating event behavior mode based on image recognition in the embodiments of the present application can be applied to an electronic device, and the electronic device includes but is not limited to mobile phones, computers, wearable devices, etc. In some other embodiments, the electronic device may also be composed of a front-end device and a back-end device. The front-end device can collect images of the event scene and then transmit the collected event scene images to the back-end device for processing.
[0057] After obtaining the event scene image by collecting images of the event scene in step S11, the event scene image can be displayed on the electronic device. Exemplarily, a front-end device, such as a mobile phone, can display the collected event scene images in real time. At the same time, the front-end device can transmit the event scene images to a back-end device, such as a server. To reduce the use of user traffic, only the thumbnail of the image can be displayed in the preview stage, while the original image can be used as the basis for subsequent image analysis and processing.
[0058] When collecting images of the event scene, images of the event scene can be collected from all aspects, globally and locally, to ensure comprehensive collection of the event scene images, thereby ensuring the accuracy of the finally speculated event behavior mode. For example, when the event scene is a room, panoramic pictures of the room, close-up images of footprints, items, etc. can be taken.
[0059] For step S12, please refer to Figure 2 As shown, it may include the following sub-steps:
[0060] S121: Perform bilateral filtering processing on the event scene image to obtain a filtered image after filtering processing.
[0061] S122: Perform edge detection processing on the filtered image to obtain an edge detection image.
[0062] S123: Extract the first image feature data of each pixel point in the edge detection image.
[0063] S124: Use the first image feature data as the image feature data of the event scene image.
[0064] Step S121 performing bilateral filtering processing on the event scene image is essentially performing filtering and noise reduction processing on the event scene image. If after the filtering process, the important information in the event scene image still cannot be accurately distinguished, then edge extraction processing can be performed to further eliminate the unnecessary information in the image and highlight the image feature information. If the information in the image is already very obvious after the filtering process, then the edge extraction process can be ignored. Because in the process of converting the image to grayscale, the edge extraction operation may cause the loss of pixel information of some necessary points. Therefore, it is necessary to clarify whether edge extraction processing is necessary.
[0065] Therefore, in step S122, when it is determined that edge detection processing needs to be performed on the filtered image, the filtered image can be subjected to edge detection processing to obtain an edge detection image. If it is determined that edge detection processing is not required for the filtered image, then the second image feature data of each pixel point in the filtered image can be extracted; and the second image feature data is used as the image feature data of the event scene image.
[0066] In an alternative embodiment, timing can be started when the filtered image is obtained. If an edge detection instruction is received within a preset time period, it is determined that edge detection processing needs to be performed on the filtered image; otherwise, it can be determined that edge detection processing is not required for the filtered image. It can be understood that the specific value of the preset time period here can be flexibly set by the developer.
[0067] In another alternative embodiment, the event scene image and the filtered image can be respectively grayscaled to obtain an event scene grayscale image and a filtered grayscale image. When the coincidence degree between the event scene grayscale image and the filtered grayscale image is less than a preset coincidence degree threshold, it is determined that edge detection processing needs to be performed on the filtered image. If the coincidence degree between the event scene grayscale image and the filtered grayscale image is greater than or equal to the preset coincidence degree threshold, it indicates that the main feature information has been extracted and the recognition rate is relatively high, and it can be determined that edge detection processing is not required for the filtered image at this time. The preset coincidence degree threshold here can be flexibly set by the developer, for example, it can be set to 90%.
[0068] It should be noted that the edge detection instruction in the embodiments of the present application can be issued by the user. For example, the user can decide whether to perform edge detection processing on the filtered image based on important information in the event scene image or the filtered image, such as whether the evidence in the image can be clearly identified.
[0069] In some embodiments, the electronic device can also determine whether the evidence in the filtered image can be clearly identified. If it is confirmed that it can be identified, the second image feature data of each pixel point in the filtered image can be directly extracted, and the second image feature data is used as the image feature data of the event scene image and input into a preset evidence recognition model to obtain the evidence category information of the evidence in the event scene image. If it is confirmed that it cannot be identified, edge detection processing can be performed on the filtered image to obtain an edge detection image, the first image feature data of each pixel point in the edge detection image is extracted, and the first image feature data is used as the image feature data of the event scene image and input into a preset evidence recognition model to obtain the evidence category information of the evidence in the event scene image.
[0070] After obtaining the filtered image, the contour edges in the filtered image can be recognized to determine whether the evidence in the filtered image can be clearly identified. For example, when the clarity of each edge contour in the filtered image is greater than or equal to a preset clarity threshold, it is determined that the evidence in the filtered image can be clearly identified. When the clarity of each edge contour in the filtered image is less than the preset clarity threshold, it is determined that the evidence in the filtered image cannot be clearly identified.
[0071] Next, the specific process of performing bilateral filtering on the event scene image in step S121 will be described.
[0072] The bilateral filtering algorithm is a non-linear algorithm. While this algorithm performs noise reduction processing to ensure that the image effect is smoother, it can also completely preserve the image edges, avoiding the situation where the edge blurring causes the image feature extraction result to be unclear. Compared with general filtering algorithms, the bilateral filtering algorithm can use not only the pixel position information as the calculation basis, but also the image feature information of pixel points as the calculation basis, for example, analyzing and calculating based on aspects such as image pixel similarity, brightness, and color. The formula of the bilateral filtering algorithm in the embodiments of the present application is as follows:
[0073]
[0074] Among them, h(x) represents the second image feature data corresponding to pixel point x after bilateral filtering processing. x1 represents a pixel point adjacent to pixel point x. The functions f(x) and f(x1) respectively represent the image feature data corresponding to pixel point x and pixel point x1 in the image before the bilateral filtering processing algorithm, that is, the image feature data corresponding to pixel point x and pixel point x1 in the event scene image respectively. The c function represents the difference in spatial geometry between different pixel points, and the s function represents the difference in image feature information between different pixel points. For example, it represents the difference information in photometric and / or color difference. The k function is a normalization function.
[0075] Through the bilateral filtering operation, the noise pixels in the event scene image can be removed, while the parts with large pixel differences are retained. For example, trace-related evidence can retain edge traces, such as footprints, tire tracks, etc., and item-related evidence can retain the item outline, such as cars, etc.
[0076] When processing the event scene image through the above bilateral filtering algorithm, weighted processing can be performed according to two aspects: spatial information and picture pixel information, so as to calculate the new pixel information of the current pixel point.
[0077] It should be noted that in the embodiment of the present application, after obtaining the event scene image, the image type of the event scene image can be determined, and then the weighted information corresponding to the image type can be determined according to the preset correspondence table between image type and weighted information. The preset correspondence table between image type and weighted information includes the correspondence between the image type and the spatial information weighting value and the pixel information weighting value. In this way, the spatial information weighting value and the pixel information weighting value corresponding to the image type can be determined, and then the event scene image can be subjected to bilateral filtering processing based on the spatial information weighting value and the pixel information weighting value to obtain the filtered image after filtering processing.
[0078] The smaller the spatial information weighting value, the clearer the picture. The larger the spatial information weighting value, the blurrier the picture. The smaller the pixel information weighting value, the clearer the edge. The larger the pixel information weighting value, the blurrier the edge. Since the edges and pixel diversity of item images are rich, for item images, the spatial information weighting value can be set to a = 6, and the pixel information weighting value can be set to b = 24 to ensure a clear edge while ensuring the clarity of the picture. For trace images, the image edges and pixel information are relatively single, and there are cases where scratches are not obvious. The spatial information weighting value can be set to a = 2, and the pixel information weighting value can be set to b = 12 to make the distance and difference between the image pixels and the neighboring pixels smaller and try to ensure that the traces are obvious.
[0079] During the process of bilateral filtering of the event scene image, corresponding spatial information weighting values and pixel information weighting values can be selected based on the type of the event scene image and the preset corresponding relationship table to process the event scene image, ensuring the filtering effect. The specific filtering process is as follows:
[0080] Import the image to be processed and digitize the image information. The main purpose of this step is to convert the pixel information of the image into an array. Starting from the first point, traverse and perform double weighting of space and pixels. During the weighting process, the base numbers of the spatial and pixel weighting values are default values determined according to the image type, which are also the values determined according to the type of the event scene image and the preset corresponding relationship table. Weight each pixel point to generate a new result. Finally, generate an image with the same size as the original image.
[0081] The larger the spatial information weighting value a, the smoother the image. The smaller a is, the greater the weight of the center point and the smaller the weight of the surrounding area, and the smaller the filtering effect. The larger the pixel information weighting value b, the more blurred the edge. The smaller b is, the clearer the edge. Therefore, in the embodiments of the present application, developers can flexibly set the corresponding spatial information weighting value and pixel information weighting value according to the image type.
[0082] Next, the process of edge detection processing will be described.
[0083] Since there has been a filtering operation before, the filtering process can be skipped during the edge detection process. During the edge detection process, in order to reduce the situation of data loss and excessive edge detection errors, the filtered image can be first converted into a grayscale image. There are generally RGB weighted average and RGB maximum grayscale methods for grayscale processing. Relatively speaking, the weighted average method considers the sensitivity of the human eye to different colors during the grayscale process, unifies the weight values of each channel, and can better retain the general shape and color information of the image. Therefore, compared with the maximum value method, it can improve the accuracy of edge detection more. So preferably, the weighted average method can be used for grayscale processing.
[0084] After completing the grayscale operation, perform gradient processing on the corresponding pixel array to calculate the gradient values of each pixel point in the x and y directions in the grayscale image. For example, the Sobel operator of the Canny algorithm can be used to calculate the gradient values of each pixel point.
[0085] After completing the gradient processing, enter the non-maximum suppression step. In this step, compare the gradient of the current pixel in the preset gradient directions, such as comparing the gradients in the four directions of 0 degrees, 45 degrees, 90 degrees, and 135 degrees, and leave the pixel point corresponding to the maximum gradient as the edge pixel point.
[0086] After non-maximum suppression processing, the edge pixels of the image are already relatively clear. To minimize the interference of noise information near the edge, double-threshold processing can be used to further reduce the noise information. The process is as follows:
[0087] Set high and low thresholds: G h , G l , (G l < G h );
[0088] Calculate the gradient value of the current pixel p: G p ;
[0089] Judge the relationship between G p , G h , G l :
[0090] If G p is greater than G h , it is considered a strong edge;
[0091] If G p is greater than G l and less than G h , it is considered a weak edge;
[0092] If G p is less than G l , the pixel will be eliminated.
[0093] Further eliminate the pixel points on the weak edge line through the geometric distance between the weak edge pixels and the strong edge pixels. Assume a weak edge pixel point p. If there is a strong edge pixel in any of the 8 neighboring pixels around the pixel point p, the weak edge pixel point is considered a real edge.
[0094] In the embodiment of the present application, pixel screening is further completed by setting two high and low thresholds. The low threshold is fixed. The higher the high threshold, the fewer details. The higher the fixed high threshold, the lower the low threshold, and the more details.
[0095] It can be understood that before inputting the image feature data into the preset evidence recognition model, the following steps can also be included:
[0096] Obtain a training sample image set; the training sample image set includes multiple sample images, and corresponding evidence category information is marked for the evidence in each sample image;
[0097] Train an evidence recognition model based on the training sample image set and the preset training model.
[0098] Specifically, a set of on-site images of historical events can be obtained, and bilateral filtering and edge detection are sequentially performed on each on-site image of historical events in the set of on-site images of historical events to obtain historical edge detection images. The categories of evidence in each historical edge detection image are labeled to obtain sample images, and the set of each sample image is used as a training sample image set. It should be noted that, according to the attention to important information in the picture, the categories of evidence in each historical edge detection image can be labeled, such as fingerprints, footprints, vehicles, etc. During the training process, in order to ensure data diversity and richness, operations such as rotating and offsetting the images can also be performed to ensure the accuracy of the training results.
[0099] The process of sequentially performing bilateral filtering and edge detection on each on-site image of historical events can refer to the above introduction and will not be elaborated here. By performing bilateral filtering and edge detection on the image, the accuracy and robustness of subsequent algorithms are improved, and at the same time, the edge and contour information in the image can be effectively detected, providing basic data for subsequent object detection and segmentation.
[0100] The preset training model in the embodiments of the present application can be a Mask R-CNN training model. Since there may be multiple important evidence information in the collected pictures, and considering that there are many items to be recognized and accurate boundary positioning and instance segmentation are required, the Mask R-CNN model can be used as the initial model.
[0101] In some embodiments, after inputting the image feature data into the preset evidence recognition model to obtain the evidence category information of the evidence in the on-site image of the event, the evidence in the on-site image of the event can also be labeled according to the evidence category information output by the evidence recognition model, and the on-site image of the event labeled with the evidence category information is added to the training sample image set to obtain a new training sample image set. Based on the new training sample image set, a new evidence recognition model is trained. When obtaining the image feature data corresponding to the new on-site image of the event, the image feature data corresponding to the new on-site image of the event is input into the new evidence recognition model to obtain the evidence category information of the evidence in the new on-site image of the event. That is, after obtaining the on-site image of the event, image processing can be completed based on the on-site image of the event, the specific information in the image can be recognized through deep learning, and the model can be updated based on the on-site image of the event, further improving the reliability of the model.
[0102] The evidence feature information of the evidence in the embodiments of the present application includes but is not limited to the position information, status information, etc. of the evidence. Here, the position can be the position where the evidence itself is located, or the position relationship between the evidence and other evidence. Here, the status can be whether the evidence is damaged, whether it is covered by other evidence, etc.
[0103] In step S15, the evidence feature information of the evidence can be obtained by analyzing the images of the event scene. For example, after obtaining the edge detection image, the content in the edge detection image can be recognized and analyzed to obtain the evidence feature information of the evidence.
[0104] The basic event information in the embodiments of the present application includes, but is not limited to, the type information of the event and the scene information of the scene where the event occurs. The type information can indicate the event type to which the event belongs. The scene information includes, but is not limited to, the category of the scene where the event occurs, such as occurring indoors, occurring outdoors, etc.
[0105] Step S16 may include the following steps:
[0106] Match the target evidence event record with the highest comprehensive matching degree of the evidence type information, the evidence feature information, and the basic event information in a preset database; the database stores multiple evidence event records, and each evidence event record includes historical evidence type information, historical evidence feature information, historical event basic information, and the corresponding historical event behavior mode;
[0107] Speculate the event behavior mode of the event scene based on the historical event behavior mode in the target evidence event record.
[0108] It should be noted that the target evidence event record can be sourced from real case data and public data sets. The existing historical evidence type information, historical evidence feature information, historical event basic information, and historical event behavior mode can be sorted and classified to construct a database.
[0109] In specific applications, the evidence type information, evidence feature information, and basic event information corresponding to the current event scene can be matched in this database to find the target evidence event record with the highest comprehensive matching degree with the current event scene.
[0110] For example, for each evidence event record in the database, the first similarity between the evidence type information corresponding to the current event scene and the historical evidence type information in the evidence event record, the second similarity between the evidence feature information corresponding to the current event scene and the historical evidence feature information in the evidence event record, and the third similarity between the basic event information corresponding to the current event scene and the historical event basic information in the evidence event record can be calculated, and the comprehensive matching degree corresponding to the evidence event record can be calculated based on the first similarity, the second similarity, and the third similarity. In some embodiments, corresponding weights can also be preset for the evidence type information, the evidence feature information, and the basic event information, and the comprehensive matching degree corresponding to the evidence event record can be calculated based on the corresponding weights and the above first similarity, second similarity, and third similarity.
[0111] The event behavior modes in the embodiments of the present application include, but are not limited to, the acting process and acting path based on evidence, etc.
[0112] In some embodiments, after inferring the event behavior mode, a corresponding appraisal report can also be generated according to the event behavior mode in combination with the event-based reasoning information and presented to the user. For example, according to the identified evidence type, combined with the basic reasoning information associated with the event such as the event occurrence location, event occurrence time, and the inferred event behavior mode of "breaking xx with xx tool", the following appraisal report is generated:
[0113] On xx date, xx month, xxxx year, in the afternoon, an xx event was found at xx location. According to the footprints at the scene, it is speculated that there were xx people, and xx tool was used to break xx, etc.
[0114] In some embodiments, the user can also be prompted according to the event behavior mode, such as prompting relevant personnel to focus on key areas and monitoring time, etc.
[0115] In the embodiments of the present application, bilateral filtering can remove noise and texture information in the image while retaining edge information, which helps to improve the accuracy and robustness of subsequent algorithms. Edge detection can effectively detect edge and contour information in the image, providing basic data for subsequent object detection and segmentation. Mask R-CNN is an efficient deep learning algorithm that can simultaneously complete object detection and instance segmentation and has achieved good results, and can be used for tasks such as image recognition and evidence tracking.
[0116] It should be understood that although the steps in the above flowchart are sequentially shown according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.
[0117] Embodiment 2:
[0118] Based on the same inventive concept, the embodiments of the present application provide an electronic device, which includes a processor 301 and a memory 302. A computer program is stored in the memory 302. The processor 301 and the memory 302 are communicatively connected through a communication bus. The processor 301 executes the computer program to implement the steps of the method in Embodiment 1 above, which will not be elaborated here. It can be understood,Figure 3 The structure shown is only schematic, and the electronic device may also include more or fewer components than those shown in Figure 3 , or have a configuration different from that shown in Figure 3 .
[0119] The processor 301 can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 301 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.
[0120] The memory 302 can include, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), etc.
[0121] This embodiment also provides a computer-readable storage medium, such as a floppy disk, an optical disc, a hard disk, a flash memory, a USB flash drive, an SD card, an MMC card, etc. One or more programs for implementing the above steps are stored in the computer storage medium, and these one or more programs can be executed by one or more processors 301 to implement the steps of the method in the first embodiment above, which will not be elaborated here.
[0122] It should be noted that the diagrams provided in this embodiment only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The types, quantities and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex. The structures, proportions, sizes, etc. shown in the diagrams of this specification are only used to cooperate with the content disclosed in the specification for those familiar with this technology to understand and read, and are not used to limit the limiting conditions under which the present invention can be implemented. Therefore, they do not have technical substance. Any modification of the structure, change of the proportional relationship or adjustment of the size, without affecting the effects that the present invention can produce and the purposes that can be achieved, should still fall within the scope that can be covered by the technical content disclosed in the present invention. At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" cited in this specification are only for the convenience of clear narration, rather than used to limit the scope in which the present invention can be implemented. The change or adjustment of their relative relationships, without substantial change of the technical content, should also be regarded as the scope in which the present invention can be implemented.
[0123] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification. The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.
Claims
1. A method for inferring event behavior based on image recognition, characterized in that: include: Capturing images of the event scene to obtain an event scene image; Performing feature extraction processing on the event scene image to obtain image feature data of the event scene image; Inputting the image feature data into a preset evidence recognition model to obtain evidence category information of the evidence in the event scene image; Obtaining evidence feature information of the evidence in the event scene image; Acquire basic event information of the event associated with the event scene image; The basic event information includes the type information of the event and the scene information of the scene where the event occurs; The event behavior pattern at the event scene is inferred based on the evidence category information, the evidence feature information and the event basic information.
2. The event behavior mode inference method based on image recognition according to claim 1, characterized in that: The step of performing feature extraction processing on the event scene image to obtain image feature data of the event scene image includes: Performing bilateral filtering on the event scene image to obtain a filtered image; Performing edge detection processing on the filtered image to obtain an edge detection image; Extracting first image feature data of each pixel in the edge detection image; The first image feature data is used as image feature data of the event scene image.
3. The event behavior mode inference method based on image recognition as claimed in claim 2, characterized in that: The step of performing edge detection processing on the filtered image to obtain an edge detected image includes: When it is determined that the filtered image needs to be subjected to edge detection processing, edge detection processing is performed on the filtered image to obtain an edge detected image.
4. The event behavior mode inference method based on image recognition as claimed in claim 3, characterized in that: The determining that edge detection processing needs to be performed on the filtered image includes: When the filtered image is obtained, timing is started, and if an edge detection instruction is received within a preset timing period, it is determined that edge detection processing needs to be performed on the filtered image; and / or, The event scene image and the filtered image are respectively grayed to obtain an event scene gray image and a filtered gray image. When the degree of overlap between the event scene gray image and the filtered gray image is less than a preset overlap threshold, it is determined that edge detection processing needs to be performed on the filtered image.
5. The event behavior mode inference method based on image recognition as claimed in claim 2, characterized in that: The step of performing bilateral filtering on the event scene image to obtain a filtered image includes: Determining the image type of the event scene image; Determine the spatial information weighted value and the pixel information weighted value corresponding to the image type according to a preset image type and weighted information correspondence table; the preset image type and weighted information correspondence table contains the correspondence between the image type and the spatial information weighted value and the pixel information weighted value; Based on the spatial information weighted value and the pixel information weighted value, bilateral filtering is performed on the event scene image to obtain a filtered image after filtering.
6. The event behavior mode inference method based on image recognition according to claim 1, characterized in that: Before inputting the image feature data into a preset evidence recognition model, the method includes: Acquire a collection of images from the scene of historical events; performing bilateral filtering and edge detection processing on each historical event scene image in the historical event scene image set in sequence to obtain a historical edge detection image; Labeling the category of evidence in each of the historical edge detection images to obtain a sample image, and using a set of the sample images as a training sample image set; The evidence recognition model is obtained by training based on the training sample image set and the preset training model.
7. The event behavior mode inference method based on image recognition according to claim 6, characterized in that: After inputting the image feature data into a preset evidence recognition model to obtain evidence category information of the evidence in the event scene image, the method includes: Marking the evidence in the event scene image according to the evidence category information; Adding the event scene image annotated with the evidence category information to the training sample image set to obtain a new training sample image set; Performing model training based on the new training sample image set to obtain a new evidence recognition model; When the image feature data corresponding to the new event scene image is obtained, the image feature data corresponding to the new event scene image is input into the new evidence recognition model to obtain the evidence category information of the evidence in the new event scene image.
8. The event behavior mode inference method based on image recognition according to any one of claims 1 to 7, characterized in that: The inferring the event behavior mode at the event scene based on the evidence category information, the evidence feature information and the event basic information includes: Matching a target evidence event record with the highest comprehensive matching degree with the evidence type information, the evidence feature information and the event basic information in a preset database; the database stores a plurality of evidence event records, each of which includes historical evidence type information, historical evidence feature information, historical event basic information and corresponding historical event behavior mode; The event behavior pattern at the event scene is inferred based on the historical event behavior pattern in the target evidence event record.
9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 8 is implemented.