Illegal behavior identification method, device, equipment, storage medium and product
By combining image and sound feature data to identify illegal behaviors during land acquisition and demolition, using drones and camera equipment to collect data in real time, the problems of low identification accuracy and efficiency in the existing technology are solved, and more efficient monitoring of illegal behaviors is achieved .
Patent Information
- Application Number
- CN202410656262.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-24
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-05-24
AI Technical Summary
The accuracy and efficiency of identifying illegal behaviors during land acquisition and demolition are low, mainly because manual video viewing takes a long time and low accuracy of time-point identification alone.
By obtaining horizontal and vertical images and sound data of the area to be demolished based on the demolition announcement information, pre-processing and feature extraction, combining image and sound feature data to identify illegal behaviors, using drones and camera equipment to collect data in real time, adjusting cruise paths and Camera device location to improve identification accuracy.
It effectively improves the accuracy and efficiency of identification of illegal acts, can promptly detect and record illegal acts, reduce manual intervention, and improve the progress and quality of demolition.
Smart Images

Figure CN118521927B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of land acquisition and demolition technology, and in particular to illegal behavior identification methods, devices, equipment, storage media and products. Background Art
[0002] With the acceleration of urbanization, land acquisition and demolition work is becoming more and more frequent. However, in the process of land acquisition and demolition, some illegal acts often occur, such as unauthorized construction, over-standard construction, illegal occupation of land, etc. These illegal acts not only affect the progress and quality of land acquisition and demolition, but also bring hidden dangers to social stability. In order to timely discover the above-mentioned illegal acts, the commonly used method is to use cameras for 24-hour uninterrupted monitoring, and then determine whether there are illegal acts through manual review of the video. However, manual review of the video requires a lot of time and human resources, and the accuracy of identifying illegal acts based on the video at a certain point in time is low. Therefore, the accuracy and efficiency of identifying illegal acts in the above method are low.
[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention
[0004] The main purpose of this application is to provide a method, device, equipment, storage medium and product for identifying illegal activities, aiming to solve the technical problems of low accuracy and efficiency in identifying illegal activities in the prior art.
[0005] To achieve the above purpose, the present application proposes a method for identifying illegal behavior, the method comprising:
[0006] Determine the area to be demolished according to the current demolition notice information, and obtain the horizontal image, vertical image and sound data of the area to be demolished;
[0007] Preprocessing the transverse image and the longitudinal image, and performing image recognition on the preprocessed images;
[0008] Extracting features from the sound data to obtain current sound feature data;
[0009] Determine whether there is any illegal behavior in the area to be demolished based on the image recognition result and / or the current sound feature data
[0010] In one embodiment, the step of determining the area to be demolished according to the current demolition notice information and acquiring the horizontal image, the vertical image and the sound data of the area to be demolished includes:
[0011] Performing text recognition on the current demolition notice information to obtain the demolition notice text;
[0012] Searching the demolition notice text to obtain the geographical location representation text;
[0013] Determine each location endpoint according to the geographic location characterization text, and connect adjacent endpoints of each location endpoint in sequence to obtain an area to be demolished;
[0014] Dividing the area to be demolished into a shielded area and a non-shielded area;
[0015] Setting a preset type of camera equipment at key checkpoints and key areas of the shielded area, and setting a cruise path according to the non-shielded area;
[0016] Acquire environmental information of the area to be demolished, and determine the working mode of the camera device according to the environmental information;
[0017] Acquiring lateral image and sound data of the area to be demolished by means of the camera device in the working mode;
[0018] The drone is controlled to fly along the cruise path to collect a longitudinal image of the area to be demolished.
[0019] In one embodiment, the step of preprocessing the horizontal image and the vertical image and performing image recognition on the preprocessed images includes:
[0020] grayscale the transverse image and the longitudinal image using a weighted average algorithm;
[0021] The lateral and longitudinal images after grayscale processing are subjected to noise reduction by using the Wiener filtering algorithm;
[0022] Determine the image recognition dimensions based on the illegal behavior measurement criteria;
[0023] Perform edge detection on the denoised horizontal image and the vertical image according to the image recognition dimension to obtain the grayscale value of the pixels on both sides of the edge, the relative distance from the edge point to the center point, and the direction angle of the edge point;
[0024] Dividing the de-noised horizontal image and vertical image into a ground image, a building image and a vehicle image according to the grayscale values of the pixels on both sides of the edge, the relative distance and the direction angle of the edge point;
[0025] The ground image, the building image and the vehicle image are recognized respectively.
[0026] In one embodiment, the step of extracting features from the sound data to obtain current sound feature data includes:
[0027] Performing noise reduction on the sound data;
[0028] Performing standardization processing on the noise-reduced sound data to obtain standard sound data;
[0029] Performing short-time frame processing on the standard sound data;
[0030] Perform Fourier transform on short-time frame sound data through short-time Fourier transform algorithm to obtain sound spectrum information;
[0031] Extracting features from the sound spectrum information to obtain current spectrum feature data;
[0032] Calculating the amplitude value of the short-time frame sound data, and determining the current amplitude characteristic data according to the amplitude value;
[0033] Determine the total length of the standard sound data, the start time of each sound, and the end time of each sound;
[0034] Determine the current duration feature data according to the total length, the start time of each sound, and the end time of each sound;
[0035] The current frequency spectrum feature data, the current amplitude feature data and the current duration feature data are normalized to obtain current sound feature data.
[0036] In one embodiment, the step of determining whether there is an illegal act in the area to be demolished based on the image recognition result and / or the current sound feature data includes:
[0037] Obtaining a ground recognition result, a building recognition result, and a vehicle recognition result at the current moment according to the image recognition result;
[0038] When the ground recognition result shows that there are tire marks of a preset earth-breaking vehicle, and / or the building recognition result shows that there is a construction object, and / or the vehicle recognition result shows that there is a preset earth-breaking vehicle, it is determined that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished;
[0039] Get the building recognition result at the last moment;
[0040] Determine the images to be compared at the same time point on different dates in the same area according to the building recognition result at the current moment and the building recognition result at the previous moment;
[0041] Performing comparison analysis on the images to be compared;
[0042] Comparing the current sound feature data with preset environmental sound feature data;
[0043] When the image comparison analysis result indicates that the construction object has changed and / or the sound feature comparison result indicates that preset sound data exists, it is determined that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished.
[0044] In one embodiment, after the step of determining that there is an illegal act in the area to be demolished according to the image recognition result and / or the current sound feature data, the method further includes:
[0045] Determine the object of the violation that caused the violation;
[0046] When the number of the illegal objects is greater than or equal to a preset number threshold, performing multi-target tracking on the plurality of illegal objects to obtain a current tracking path;
[0047] determining a violation area according to the current tracking path;
[0048] Adjust the drone's cruising path according to the illegal area;
[0049] The position of the camera device is adjusted according to the illegal area.
[0050] In addition, to achieve the above purpose, the present application also proposes an illegal behavior identification device, the illegal behavior identification device comprising:
[0051] An acquisition module, used to determine the area to be demolished according to the current demolition notice information, and acquire the horizontal image, vertical image and sound data of the area to be demolished;
[0052] A processing module, used for preprocessing the horizontal image and the vertical image, and performing image recognition on the preprocessed images;
[0053] A feature extraction module, used to extract features from the sound data to obtain current sound feature data;
[0054] A determination module is used to determine whether there is any illegal behavior in the area to be demolished based on the image recognition result and / or the current sound feature data.
[0055] In addition, to achieve the above-mentioned purpose, the present application also proposes an illegal behavior identification device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the illegal behavior identification method described above.
[0056] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the illegal behavior identification method described above are implemented.
[0057] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the illegal behavior identification method described above are implemented.
[0058] One or more technical solutions proposed in the present application have at least the following technical effects: determining the area to be demolished according to the current demolition notice information, and obtaining the horizontal image, vertical image and sound data of the area to be demolished; preprocessing the horizontal image and the vertical image, and performing image recognition on the preprocessed images; performing feature extraction on the sound data to obtain current sound feature data; determining the existence of illegal acts in the area to be demolished according to the image recognition results and / or the current sound feature data; through the above method, after determining the area to be demolished, determining the image recognition results and the current sound feature data according to multi-dimensional parameters such as the horizontal image, the vertical image and the sound data, and using the image recognition results and the sound feature data to comprehensively determine the existence of illegal acts in the area to be demolished, thereby effectively improving the accuracy and efficiency of identifying illegal acts. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0061] Figure 1 A flowchart of the first embodiment of the method for identifying illegal behavior of this application is provided;
[0062] Figure 2 A flowchart diagram of the second embodiment of the method for identifying illegal behavior of this application;
[0063] Figure 3 This is a schematic diagram of the module structure of the illegal behavior identification device according to the embodiment of the present application;
[0064] Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the illegal behavior identification method in the embodiment of the present application.
[0065] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0066] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an identification controller, etc. The following takes the identification controller as an example to illustrate this embodiment and the following embodiments.
[0067] Based on this, the present application embodiment provides a method for identifying illegal behavior. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the illegal behavior identification method of the present application.
[0068] In this embodiment, the illegal behavior identification method includes steps S10 to S40:
[0069] Step S10, determining the area to be demolished according to the current demolition notice information, and obtaining the horizontal image, vertical image and sound data of the area to be demolished.
[0070] It should be understood that the current demolition announcement information refers to the announcement information related to the demolition that is published by an authoritative agency and notified to residents. The current demolition announcement information includes but is not limited to information such as demolition time, location, demolition area and specific demolition implementation plan. The area to be demolished refers to an area that requires normal demolition, for example, a certain community. The horizontal image represents the image from the plane dimension, for example, the regional image from left to right, and the vertical image represents the image from the spatial dimension, for example, the regional image from top to bottom. The sound data refers to the data generated by the sound in the area to be demolished.
[0071] Further, step S10 includes: performing text recognition on the current demolition notice information to obtain demolition notice text; searching the demolition notice text to obtain geographic location representation text; determining each location endpoint according to the geographic location representation text, and connecting adjacent endpoints of each location endpoint in sequence to obtain the area to be demolished; dividing the area to be demolished into an obstructed area and a non-obstructed area; setting a preset type of camera equipment at key checkpoints and key areas of the obstructed area, and setting a cruise path according to the non-obstructed area; acquiring environmental information of the area to be demolished, and determining a working mode of the camera equipment according to the environmental information; acquiring horizontal images and sound data of the area to be demolished through the camera equipment in the working mode; and controlling the drone to fly along the cruise path to collect vertical images of the area to be demolished.
[0072] It can be understood that the text of the demolition notice refers to all the text in the current demolition notice information, and the geographic location representation text refers to the text related to the geographic location, for example, the demolition address is xx, and then the adjacent endpoints of each location endpoint are connected in sequence to form the area to be demolished. In order to effectively improve the accuracy of image acquisition, the area to be demolished needs to be divided into blocked areas and non-blocked areas, where the blocked area is the blind spot of the drone. At this time, it is necessary to set preset types of camera equipment at the key checkpoints and key areas in the blocked area. The key checkpoint can be the only intersection in and out of the area to be demolished. The camera equipment can be a camera with radio and infrared functions. The cruise path refers to the path for controlling the drone to cruise. The area covered by the drone when flying along the cruise path happens to completely include the area to be demolished, so as to make up for the defects of insufficient number of camera equipment and blind spots.
[0073] It should be understood that in this embodiment, the working mode of the camera equipment will be set according to the environmental information of the area to be demolished. For example, in severe haze or heavy rain, it will be set to the infrared working mode to enhance the image quality. The camera equipment is then used to obtain the lateral image and sound data of the area to be demolished, and the drone is controlled to fly according to the cruising path, and the longitudinal image of the area to be demolished is collected in real time during the flight.
[0074] Step S20, preprocessing the horizontal image and the vertical image, and performing image recognition on the preprocessed images.
[0075] It should be understood that after acquiring the horizontal image and the vertical image, the horizontal image and the vertical image need to be preprocessed respectively. The preprocessing includes but is not limited to grayscale processing, noise reduction, and edge detection. After the processing is completed, the preprocessed image needs to be subjected to image recognition to determine whether there are any illegal acts that hinder the normal land acquisition and demolition in the area to be demolished.
[0076] Furthermore, step S20 includes: gray-scaling the horizontal image and the vertical image through a weighted average algorithm; denoising the gray-scaled horizontal image and the vertical image through a Wiener filtering algorithm; determining the image recognition dimension according to the violation measurement standard; performing edge detection on the denoised horizontal image and the vertical image according to the image recognition dimension to obtain the grayscale value of the pixels on both sides of the edge, the relative distance from the edge point to the center point, and the direction angle of the edge point; dividing the denoised horizontal image and the vertical image into a ground image, a building image, and a vehicle image according to the grayscale value of the pixels on both sides of the edge, the relative distance, and the direction angle of the edge point; and identifying the ground image, the building image, and the vehicle image, respectively.
[0077] It is understandable that after acquiring the horizontal image and the vertical image, in order to retain more image details and ensure image clarity, the weighted average algorithm is used to grayscale the horizontal image and the vertical image respectively. Specifically, the RGB component of each pixel in the horizontal image and the vertical image is multiplied by the corresponding weight and averaged. The weight is designed according to the sensitivity of the human body to the three primary colors of red, green and blue, and is in line with human visual perception. In addition, in order to effectively improve the noise reduction effect, the Wiener filtering algorithm is used to reduce the noise of the grayscale horizontal image and the vertical image. In the noise reduction process, the Wiener filter is also needed to use the local image and noise mean square error calculation. When the local mean square error is the largest, the noise can be distinguished and removed.
[0078] It should be understood that the image recognition dimension refers to the dimension for image recognition, which includes but is not limited to the dimensions of ground, building and vehicle. The image recognition dimension is then used to perform edge detection on the denoised horizontal and vertical images, and the denoised horizontal and vertical images are divided at the same time using the grayscale values of the pixels on both sides of the detected edges, the relative distance and the direction angle of the edge points to obtain ground images, building images and vehicle images.
[0079] Step S30, extracting features from the sound data to obtain current sound feature data.
[0080] It can be understood that the current sound feature data includes but is not limited to spectrum feature data, amplitude feature data, and duration feature data. After obtaining the current sound feature data, in order to effectively improve the recognition accuracy, the sound data also needs to be preprocessed, which includes but is not limited to noise reduction, standardization, frame processing, etc.
[0081] Further, step S30 includes: denoising the sound data; standardizing the denoised sound data to obtain standard sound data; short-time framing the standard sound data; Fourier transforming the short-time frame sound data through a short-time Fourier transform algorithm to obtain sound spectrum information; extracting features from the sound spectrum information to obtain current spectrum feature data; calculating the amplitude value of the short-time frame sound data, and determining the current amplitude feature data based on the amplitude value; determining the total length, the start time of each sound, and the end time of each sound of the standard sound data; determining the current duration feature data based on the total length, the start time of each sound, and the end time of each sound; normalizing the current spectrum feature data, the current amplitude feature data, and the current duration feature data to obtain current sound feature data.
[0082] It should be understood that after noise reduction and standardization processing, the standard sound data also needs to be short-time frame processed, that is, the standard sound data is divided into frames, and the length of each frame can be 20 to 30ms, so as to capture the short-time changes of the sound, and then the short-time frame sound data is Fourier transformed by the short-time Fourier transform algorithm to reveal the frequency components and time domain changes of the audio, and then the sound spectrum information is feature extracted by the sound feature extraction strategy. In addition, the current amplitude feature data represents the intensity of the sound at each time point. After calculating the amplitude value of the short-time frame sound data, the current amplitude feature data is extracted by the sound feature extraction strategy, and the current duration feature data is determined according to the total length of the standard sound data, the start time of each sound and the end time of each sound. In order to further optimize, the current spectrum feature data, the current amplitude feature data and the current duration feature data need to be normalized to obtain the current sound feature data.
[0083] Step S40: determining whether there is any illegal behavior in the area to be demolished based on the image recognition result and / or the current sound feature data.
[0084] It is understandable that after obtaining the image recognition results and the current sound feature data, it is determined based on at least one of the above parameters that there are illegal acts in the area to be demolished, and the illegal acts include but are not limited to unauthorized construction, over-standard construction, illegal occupation of land, etc.
[0085] Further, step S40 includes: obtaining a ground recognition result, a building recognition result and a vehicle recognition result at the current moment according to the image recognition result; determining that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished when the ground recognition result is the presence of preset earth-breaking vehicle tire marks, and / or the building recognition result is the presence of a construction object, and / or the vehicle recognition result is the presence of a preset earth-breaking vehicle; obtaining the building recognition result at the previous moment; determining images to be compared at the same time point on different dates in the same area according to the building recognition result at the current moment and the building recognition result at the previous moment; performing comparison and analysis on the images to be compared; comparing the current sound feature data with the preset environmental sound feature data; determining that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished when the image comparison and analysis result is a change in the construction object and / or the sound feature comparison result is the presence of preset sound data.
[0086] It can be understood that after obtaining the ground recognition results, building recognition results and vehicle recognition results at the current moment, a judgment of illegal behavior is made, that is, whether the ground recognition result shows that there are preset earth-breaking vehicle tire marks, whether the building recognition result shows that there are construction objects, and whether the vehicle recognition result shows that there are preset earth-breaking vehicles. The preset earth-breaking vehicles include but are not limited to bulldozers, excavators and transport vehicles, etc., and the construction objects include but are not limited to construction personnel, bricks and concrete, etc. When it is determined that at least one of the above conditions is met, the suspicious illegal behavior is marked. In addition, since poor image quality may cause omissions in recognition, the above defects can be compensated by static image comparison, that is, the images to be compared at the same time point on different dates in the same area are determined according to the building recognition results at the current moment and the building recognition results at the previous moment. For example, on the morning of March 21, 2021 The image of area A at 10 o'clock on March 21, 2021 and the image of area A at 10 o'clock on the morning of March 22, 2021 are compared and analyzed. The processing flow of the comparative analysis includes but is not limited to registration, enhancement, wavelet transform and principal component analysis. In addition, in order to effectively improve the accuracy of recognition, a judgment method of sound dimension will be added, that is, the current sound feature data will be compared with the preset environmental sound feature data. The preset sound data is not limited to mechanical operation sound, explosion sound, etc. After obtaining the image comparison and analysis results, the image in the image comparison and analysis results will be segmented to obtain the image of the changed area, and the image of the changed area will be classified to exclude interference factors such as personnel. When it is determined that the image comparison analysis result is that the construction object has changed and the sound feature comparison result is that at least one of the preset sound data exists, a suspicious illegal behavior is marked.
[0087] It should be understood that after obtaining the mark of suspected illegal behavior, an alarm message is generated and automatically pushed to the terminal of the relevant supervisory personnel in real time to urge the relevant supervisory personnel to verify the authenticity of the illegal behavior. If it is true, the marked suspicious illegal behavior will be characterized as an illegal behavior that hinders the normal land acquisition and demolition in the area to be demolished. At the same time, detailed information such as the time, location, vehicle type, etc. of the illegal behavior will be recorded for subsequent investigation and handling.
[0088] This embodiment determines the area to be demolished according to the current demolition notice information, and obtains the horizontal image, vertical image and sound data of the area to be demolished; preprocesses the horizontal image and the vertical image, and performs image recognition on the preprocessed images; performs feature extraction on the sound data to obtain current sound feature data; determines the existence of illegal acts in the area to be demolished according to the image recognition result and / or the current sound feature data; through the above method, after determining the area to be demolished, the image recognition result and the current sound feature data are determined according to multi-dimensional parameters such as the horizontal image, the vertical image and the sound data, and the image recognition result and the sound feature data are used to comprehensively determine the existence of illegal acts in the area to be demolished, thereby effectively improving the accuracy and efficiency of identifying illegal acts.
[0089] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , after step S40, further comprising steps S501 to S505:
[0090] Step S501, determining the illegal object that caused the illegal behavior.
[0091] It should be understood that illegal objects refer to those that hinder the normal land acquisition and demolition work in the area to be demolished, and such illegal objects include but are not limited to preset vehicles and construction personnel.
[0092] Step S502: When the number of the illegal objects is greater than or equal to a preset number threshold, multi-target tracking is performed on the plurality of illegal objects to obtain a current tracking path.
[0093] It can be understood that when it is determined that the number of illegal objects is greater than or equal to the preset number threshold, it indicates that there are multiple objects that have committed illegal acts, for example, multiple preset vehicles and multiple construction workers. At this time, the illegal objects are tracked by multi-target tracking using the camera equipment and drones that have captured the illegal objects. Specifically, the camera equipment and the drone cooperate with each other, and the drone is used for tracking in areas where no camera equipment is installed, and the camera equipment is used for tracking in the blind spot of the drone. Then, the current tracking path is comprehensively determined based on the ID of the camera equipment, the current position and the flight path of the drone, thereby effectively improving the accuracy of the current tracking path.
[0094] It should be noted that if different camera equipment or drones track the same illegal object, only one alarm signal needs to be generated, and then the illegal object is tracked in real time. After the illegal object leaves automatically or is expelled by relevant supervisory personnel, the next patrol and surveillance will be carried out to avoid repeated alarms.
[0095] Step S503: determining the illegal area according to the current tracking path.
[0096] It should be understood that the illegal area refers to the area where the illegal object has committed illegal acts. The illegal area is located in the area to be demolished, and the illegal area can be composed of the edge of the area to be demolished and the current tracking path.
[0097] Step S504: adjusting the cruising path of the drone according to the illegal area.
[0098] It is understandable that illegal activities are more likely to occur in illegal areas. Therefore, it is necessary to adjust the cruising path of the drone, that is, to patrol the illegal area at a high frequency to achieve the purpose of timely detection of illegal activities.
[0099] Step S505: adjusting the position of the camera device according to the illegal area.
[0100] It should be understood that after determining the illegal area, more cameras need to be added to the illegal area to record high-violation areas more frequently. For example, cameras in other non-important areas can be moved to the illegal area to increase the number of cameras in the illegal area.
[0101] This embodiment determines the illegal objects that have caused the illegal behavior; when the number of the illegal objects is greater than or equal to a preset number threshold, multi-target tracking is performed on multiple illegal objects to obtain a current tracking path; the illegal area is determined according to the current tracking path; the cruising path of the UAV is adjusted according to the illegal area; the position of the camera is adjusted according to the illegal area; through the above-mentioned method, after determining that there is an illegal behavior in the area to be demolished and making a timely alarm, it is continued to be determined whether the number of illegal objects that have caused the illegal behavior is greater than or equal to the preset number threshold. If so, multi-target tracking is performed, and the cruising path of the UAV and the position of the camera are adjusted according to the illegal area, so as to achieve the purpose of timely and convenient detection of illegal behavior.
[0102] This application also provides a device for identifying illegal behavior. Figure 3 , the illegal behavior identification device includes:
[0103] The acquisition module 10 is used to determine the area to be demolished according to the current demolition notice information, and to acquire the horizontal image, the vertical image and the sound data of the area to be demolished.
[0104] The processing module 20 is used to preprocess the horizontal image and the vertical image, and perform image recognition on the preprocessed images.
[0105] The feature extraction module 30 is used to extract features from the sound data to obtain current sound feature data.
[0106] The determination module 40 is used to determine whether there is any illegal behavior in the area to be demolished based on the image recognition result and / or the current sound feature data.
[0107] This embodiment determines the area to be demolished according to the current demolition notice information, and obtains the horizontal image, vertical image and sound data of the area to be demolished; preprocesses the horizontal image and the vertical image, and performs image recognition on the preprocessed images; performs feature extraction on the sound data to obtain current sound feature data; determines the existence of illegal acts in the area to be demolished according to the image recognition result and / or the current sound feature data; through the above method, after determining the area to be demolished, the image recognition result and the current sound feature data are determined according to multi-dimensional parameters such as the horizontal image, the vertical image and the sound data, and the image recognition result and the sound feature data are used to comprehensively determine the existence of illegal acts in the area to be demolished, thereby effectively improving the accuracy and efficiency of identifying illegal acts.
[0108] The illegal behavior identification device provided by the present application adopts the illegal behavior identification method in the above embodiment, which can solve the technical problem of low accuracy and efficiency in identifying illegal behaviors in the prior art. Compared with the prior art, the beneficial effects of the illegal behavior identification device provided by the present application are the same as the beneficial effects of the illegal behavior identification method provided by the above embodiment, and other technical features in the illegal behavior identification device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0109] In one embodiment, the acquisition module 10 is also used to perform text recognition on the current demolition notice information to obtain the demolition notice text; search the demolition notice text to obtain the geographic location representation text; determine each location endpoint according to the geographic location representation text, and connect adjacent endpoints of each location endpoint in sequence to obtain the area to be demolished; divide the area to be demolished into an obstructed area and a non-obstructed area; set a preset type of camera equipment at the key checkpoints and key areas of the obstructed area, and set a cruise path according to the non-obstructed area; obtain environmental information of the area to be demolished, and determine the working mode of the camera equipment according to the environmental information; obtain horizontal images and sound data of the area to be demolished through the camera equipment in the working mode; control the drone to fly along the cruise path to collect the longitudinal image of the area to be demolished.
[0110] In one embodiment, the processing module 20 is also used to grayscale the horizontal image and the vertical image through a weighted average algorithm; to reduce noise on the grayscale horizontal image and the vertical image through a Wiener filtering algorithm; to determine the image recognition dimension according to the violation measurement standard; to perform edge detection on the denoised horizontal image and the vertical image according to the image recognition dimension to obtain the grayscale value of the pixels on both sides of the edge, the relative distance from the edge point to the center point, and the direction angle of the edge point; to divide the denoised horizontal image and the vertical image into a ground image, a building image, and a vehicle image according to the grayscale value of the pixels on both sides of the edge, the relative distance, and the direction angle of the edge point; and to identify the ground image, the building image, and the vehicle image, respectively.
[0111] In one embodiment, the feature extraction module 30 is also used to perform noise reduction on the sound data; perform standardization processing on the noise-reduced sound data to obtain standard sound data; perform short-time frame processing on the standard sound data; perform Fourier transform on the short-time frame sound data through a short-time Fourier transform algorithm to obtain sound spectrum information; perform feature extraction on the sound spectrum information to obtain current spectrum feature data; calculate the amplitude value of the short-time frame sound data, and determine the current amplitude feature data based on the amplitude value; determine the total length, the start time of each sound, and the end time of each sound of the standard sound data; determine the current duration feature data based on the total length, the start time of each sound, and the end time of each sound; normalize the current spectrum feature data, the current amplitude feature data, and the current duration feature data to obtain current sound feature data.
[0112] In one embodiment, the determination module 40 is also used to obtain the ground recognition result, building recognition result and vehicle recognition result at the current moment based on the image recognition result; when the ground recognition result is the presence of preset tire marks of a groundbreaking vehicle, and / or the building recognition result is the presence of a construction object, and / or the vehicle recognition result is the presence of a preset groundbreaking vehicle, determine that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished; obtain the building recognition result at the previous moment; determine the images to be compared at the same time point on different dates in the same area based on the building recognition result at the current moment and the building recognition result at the previous moment; compare and analyze the images to be compared; compare the current sound feature data with the preset environmental sound feature data; when the image comparison analysis result is a change in the construction object and / or the sound feature comparison result is the presence of preset sound data, determine that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished.
[0113] In one embodiment, the determination module 40 is also used to determine the illegal objects that caused the illegal behavior; when the number of the illegal objects is greater than or equal to a preset number threshold, multi-target tracking is performed on multiple illegal objects to obtain a current tracking path; the illegal area is determined according to the current tracking path; the cruising path of the drone is adjusted according to the illegal area; and the position of the camera device is adjusted according to the illegal area.
[0114] The present application provides an illegal behavior identification device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the illegal behavior identification method in the above-mentioned embodiment one.
[0115] Reference below Figure 4 , which shows a schematic diagram of the structure of an illegal behavior identification device suitable for implementing the embodiment of the present application. The illegal behavior identification device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The illegal behavior identification device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0116] like Figure 4As shown, the illegal behavior identification device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the illegal behavior identification device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the illegal behavior identification device to communicate with other devices wirelessly or by wire to exchange data. Although the illegal behavior identification device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.
[0117] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0118] The illegal behavior identification device provided by the present application adopts the illegal behavior identification method in the above embodiment, which can solve the technical problem of low accuracy and efficiency in identifying illegal behaviors in the prior art. Compared with the prior art, the beneficial effects of the illegal behavior identification device provided by the present application are the same as the beneficial effects of the illegal behavior identification method provided by the above embodiment, and the other technical features in the illegal behavior identification device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0119] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0120] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0121] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the illegal behavior identification method in the above-mentioned embodiment.
[0122] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-ReadOnly Memory), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.
[0123] The computer-readable storage medium may be included in the illegal behavior identification device; or may exist independently without being assembled into the illegal behavior identification device.
[0124] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0125] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0126] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0127] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned illegal behavior identification method, and can solve the technical problem of low accuracy and efficiency in identifying illegal behaviors in the prior art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the illegal behavior identification method provided by the above-mentioned embodiment, and will not be repeated here.
[0128] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned illegal behavior identification method when executed by a processor.
[0129] The computer program product provided by this application can solve the technical problem of low accuracy and efficiency in identifying illegal activities in the prior art. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as those of the illegal activity identification method provided by the above embodiment, which will not be elaborated here.
[0130] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A method for identifying illegal behavior, It is characterized in that The method comprises: Determine the area to be demolished according to the current demolition notice information, and obtain the horizontal image, vertical image and sound data of the area to be demolished; Preprocessing the transverse image and the longitudinal image, and performing image recognition on the preprocessed images; Extracting features from the sound data to obtain current sound feature data; Determining, based on the image recognition result and / or the current sound feature data, that there is an illegal act in the area to be demolished; The step of preprocessing the horizontal image and the vertical image and performing image recognition on the preprocessed images comprises: grayscale the transverse image and the longitudinal image using a weighted average algorithm; The lateral and longitudinal images after grayscale processing are subjected to noise reduction by using the Wiener filtering algorithm; Determine the image recognition dimensions based on the illegal behavior measurement criteria; Perform edge detection on the denoised horizontal image and the vertical image according to the image recognition dimension to obtain the grayscale value of the pixels on both sides of the edge, the relative distance from the edge point to the center point, and the direction angle of the edge point; Dividing the de-noised horizontal image and vertical image into a ground image, a building image and a vehicle image according to the grayscale values of the pixels on both sides of the edge, the relative distance and the direction angle of the edge point; Respectively identifying the ground image, the building image, and the vehicle image; The step of extracting features from the sound data to obtain current sound feature data comprises: Performing noise reduction on the sound data; Performing standardization processing on the noise-reduced sound data to obtain standard sound data; Performing short-time frame processing on the standard sound data; Perform Fourier transform on the short-time frame sound data through the short-time Fourier transform algorithm to obtain the sound spectrum information; Extracting features from the sound spectrum information to obtain current spectrum feature data; Calculating the amplitude value of the short-time frame sound data, and determining the current amplitude characteristic data according to the amplitude value; Determine the total length of the standard sound data, the start time of each sound, and the end time of each sound; Determine the current duration feature data according to the total length, the start time of each sound, and the end time of each sound; Normalizing the current frequency spectrum feature data, the current amplitude feature data, and the current duration feature data to obtain current sound feature data; The step of determining whether there is an illegal act in the area to be demolished based on the image recognition result and / or the current sound feature data includes: Obtaining a ground recognition result, a building recognition result, and a vehicle recognition result at the current moment according to the image recognition result; When the ground recognition result shows that there are tire marks of a preset earth-breaking vehicle, and / or the building recognition result shows that there is a construction object, and / or the vehicle recognition result shows that there is a preset earth-breaking vehicle, it is determined that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished; Get the building recognition result at the last moment; Determine the images to be compared at the same time point on different dates in the same area according to the building recognition result at the current moment and the building recognition result at the previous moment; Performing comparison analysis on the images to be compared; Comparing the current sound feature data with preset environmental sound feature data; When the image comparison analysis result indicates that the construction object has changed and / or the sound feature comparison result indicates that preset sound data exists, it is determined that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished.
2. The method according to claim 1, It is characterized in that The step of determining the area to be demolished according to the current demolition notice information and obtaining the horizontal image, the vertical image and the sound data of the area to be demolished includes: Performing text recognition on the current demolition notice information to obtain the demolition notice text; Searching the demolition notice text to obtain the geographical location representation text; Determine each location endpoint according to the geographic location characterization text, and connect adjacent endpoints of each location endpoint in sequence to obtain an area to be demolished; Dividing the area to be demolished into a shielded area and a non-shielded area; Setting a preset type of camera equipment at key checkpoints and key areas of the shielded area, and setting a cruise path according to the non-shielded area; Acquire environmental information of the area to be demolished, and determine the working mode of the camera device according to the environmental information; Acquiring lateral image and sound data of the area to be demolished by means of the camera device in the working mode; The drone is controlled to fly along the cruise path to collect a longitudinal image of the area to be demolished.
3. The method according to any one of claims 1 or 2, It is characterized in that After the step of determining that there is an illegal act in the area to be demolished according to the image recognition result and / or the current sound feature data, the method further includes: Determine the object of the violation that caused the violation; When the number of the illegal objects is greater than or equal to a preset number threshold, performing multi-target tracking on the plurality of illegal objects to obtain a current tracking path; determining a violation area according to the current tracking path; Adjust the drone's cruising path according to the illegal area; The position of the camera device is adjusted according to the illegal area.
4. A device for identifying illegal behavior, It is characterized in that The device comprises: An acquisition module, used to determine the area to be demolished according to the current demolition notice information, and acquire the horizontal image, vertical image and sound data of the area to be demolished; A processing module, used for preprocessing the horizontal image and the vertical image, and performing image recognition on the preprocessed images; A feature extraction module, used to extract features from the sound data to obtain current sound feature data; A determination module, configured to determine whether there is any illegal behavior in the area to be demolished according to the image recognition result and / or the current sound feature data; The processing module is further used to grayscale the horizontal image and the vertical image by a weighted average algorithm; perform noise reduction on the grayscaled horizontal image and the vertical image by a Wiener filtering algorithm; determine the image recognition dimension according to the illegal behavior measurement standard; perform edge detection on the denoised horizontal image and the vertical image according to the image recognition dimension to obtain the grayscale value of the pixels on both sides of the edge, the relative distance from the edge point to the center point, and the direction angle of the edge point; divide the denoised horizontal image and the vertical image into a ground image, a building image, and a vehicle image according to the grayscale value of the pixels on both sides of the edge, the relative distance, and the direction angle of the edge point; and identify the ground image, the building image, and the vehicle image respectively; The feature extraction module is also used to reduce noise on the sound data; perform standardization on the noise-reduced sound data to obtain standard sound data; perform short-time frame processing on the standard sound data; perform Fourier transform on the short-time frame sound data through a short-time Fourier transform algorithm to obtain sound spectrum information; perform feature extraction on the sound spectrum information to obtain current spectrum feature data; calculate the amplitude value of the short-time frame sound data, and determine the current amplitude feature data according to the amplitude value; determine the total length, the start time of each sound, and the end time of each sound of the standard sound data; determine the current duration feature data according to the total length, the start time of each sound, and the end time of each sound; perform normalization on the current spectrum feature data, the current amplitude feature data, and the current duration feature data to obtain current sound feature data; The determination module is also used to obtain the ground recognition result, building recognition result and vehicle recognition result at the current moment according to the image recognition result; when the ground recognition result is the presence of preset tire marks of a groundbreaking vehicle, and / or the building recognition result is the presence of a construction object, and / or the vehicle recognition result is the presence of a preset groundbreaking vehicle, determine that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished; obtain the building recognition result at the previous moment; determine the images to be compared at the same time point on different dates in the same area according to the building recognition result at the current moment and the building recognition result at the previous moment; compare and analyze the images to be compared; compare the current sound feature data with the preset environmental sound feature data; when the image comparison and analysis result is that the construction object has changed and / or the sound feature comparison result is the presence of preset sound data, determine that there is an illegal act that hinders normal land acquisition and demolition in the area to be demolished.
5. A device for identifying illegal activities, It is characterized in that The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the illegal behavior identification method according to any one of claims 1 to 3.
6. A storage medium, It is characterized in that The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the illegal behavior identification method as described in any one of claims 1 to 3 are implemented.
7. A computer program product, It is characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the illegal behavior identification method according to any one of claims 1 to 3 are implemented.
