A site management system based on unmanned aerial vehicle automatic inspection

Abstract

The application discloses a kind of based on unmanned plane automatic inspection's field management system, comprising: unmanned plane, unmanned plane control end, wireless radio frequency read-write end being arranged on unmanned plane, electronic tag and electronic tag transmission interface being arranged on safety helmet or wearable device, and the electronic tag transmission interface connects alarm unit;The unmanned plane control end passes through image acquisition module and collects field operation image, transmits image to the image processing unit of unmanned plane control end and carries out image processing, judges whether there is rule-breaking behavior, if there is rule-breaking behavior, writes alarm information to the electronic tag on the safety helmet or wearable device corresponding to rule-breaking personnel by wireless radio frequency read-write end, the alarm information is transmitted to alarm unit by electronic tag transmission interface, and sends alarm to the outside by alarm unit.The unmanned plane control end is arranged in work site, and if there is rule-breaking behavior, wearable device is sent alarm, reduces rule-breaking behavior, avoids unnecessary danger to occur.

Description

Technical Field

[0001] This invention relates to the field of on-site management technology, and more specifically to an on-site management system based on unmanned aerial vehicle (UAV) automatic inspection. Background Technology

[0002] Currently, drone-based automated patrol and monitoring technology utilizes automated airports and real-time streaming media transmission protocols to transmit video collected by drones to a backend intelligent analysis and detection system in real time. The backend server efficiently analyzes the real-time video transmitted by the drones to detect any violations at the work site. Therefore, using drones for equipment inspection significantly improves work efficiency. Furthermore, drones can reach heights inaccessible to humans and achieve precision beyond the reach of the naked eye, enhancing the accuracy of on-site inspections and comprehensively improving inspection efficiency and quality.

[0003] However, current technologies using drones for automated patrols only detect violations on-site, without addressing further processing of the violations. Existing solutions might simply warn violators via text message, but due to varying work environments, personnel may not be able to check their phones promptly or correct their violations in a timely manner, potentially leading to serious consequences. Alternatively, existing solutions might offer in-person warnings, but these methods are time-consuming, labor-intensive, and extremely inefficient. Therefore, a solution is urgently needed that integrates inspection, violation determination, and warning. Summary of the Invention

[0004] This invention provides a field management system based on unmanned aerial vehicle (UAV) automatic inspection to solve the above-mentioned problems in the prior art.

[0005] This invention provides a field management system based on unmanned aerial vehicle (UAV) automated inspection, the system comprising:

[0006] The drone, the drone control terminal, the wireless radio frequency reader / writer terminal installed on the drone, the electronic tag installed on the safety helmet or wearable device, and the electronic tag transmission interface, wherein the electronic tag transmission interface is connected to the alarm unit;

[0007] The drone control terminal acquires on-site operation images through the image acquisition module and transmits the images to the image processing unit of the drone control terminal for image processing to determine whether there is any violation. If there is a violation, alarm information is written to the electronic tag on the safety helmet or wearable device of the violator through the wireless radio frequency reader / writer. The alarm information is transmitted to the alarm unit through the electronic tag transmission interface, and the alarm unit issues an alarm to the outside.

[0008] Preferably, if the image processing unit determines that the violation is a violation of not wearing a safety helmet, the radio frequency reader / writer obtains the location of the safety helmet corresponding to the violator by reading the electronic tag through radio frequency technology; the electronic tag is equipped with a GPS positioning module to generate the location information of the electronic tag; the radio frequency reader / writer reads the location information when reading the electronic tag information.

[0009] Once the location information of the safety helmet of the violator is obtained, the wireless radio frequency reader / writer writes alarm information and location information to the electronic tag on the wearable device corresponding to the violator; the alarm information and location information are transmitted to the alarm unit through the electronic tag transmission interface, and when the alarm unit sends out an alarm, it broadcasts the location information of the safety helmet.

[0010] Preferably, the drone control terminal is equipped with a personnel information database, in which the identity information of the same personnel corresponds to two different electronic tag identifiers, and each electronic tag has a unique electronic tag identifier;

[0011] An electronic tag is set on both the safety helmet and the wearable device of the same worker; if the worker changes the safety helmet or wearable device, the updated electronic tag is displayed in the worker's information database.

[0012] Preferably, the image acquisition module includes an image acquisition device mounted on the drone, a directional camera or a mobile camera mounted at the work site;

[0013] During the inspection process, the drone sends its current location to the drone control terminal. The drone control terminal then sends acquisition and transmission commands to image acquisition modules within a preset range of the drone's location via wireless communication. The image acquisition modules within the preset range acquire image information within the current time period and transmit the image information to the drone control terminal.

[0014] The image processing unit of the UAV control terminal filters the received image information, filters out images with workers, and analyzes the violations in the filtered images.

[0015] During the analysis of violations, feature information of the image to be analyzed is extracted and matched with features in the violation database. If one or more matching features are found, the violation corresponding to that feature is marked. Each violation corresponds to a standard behavior. The text information of the violation and the standard behavior is transmitted to the alarm unit. The alarm unit converts the text information into voice information and issues a voice alarm. The standard behavior reminds the operator to correct the violation.

[0016] In the process of extracting feature information from the image to be analyzed, a bilinear convolutional neural network is constructed based on the prototype network as the network framework and the MobileNet-V2 and ResNet-18 networks as the embedding function of the prototype network to perform feature recognition on the image to be analyzed and obtain feature information.

[0017] Preferably, the UAV control terminal and the image acquisition module are connected wirelessly, and there are multiple UAV control terminals set up at the work site;

[0018] The drone control terminal is equipped with a timing unit. When the drone control terminal controls the alarm unit to issue a warning via the wireless radio frequency reader / writer, the timing unit is activated. After a preset time, the drone control terminal sends a command to the image acquisition module to re-acquire the image information of the violator. If the image analysis unit determines that the violator still has a violation, the image acquisition module acquires the facial information of the workers around the violator, and the facial recognition module identifies the identity information of the corresponding workers. Based on the identity information, the corresponding electronic tag of the worker is obtained, and a reminder command is written to the corresponding electronic tag via the wireless radio frequency reader / writer on the drone. The reminder command includes the violator's name, employee number, violation, and compliant behavior. The alarm unit on the safety helmet or wearable device of the reminded worker sounds an alarm and broadcasts the reminder command via voice, so that the reminded worker can help the violator correct the violation in a timely manner.

[0019] Preferably, the UAV control terminal also includes a statistical model;

[0020] Locations where the probability of a violation is predicted to be higher than a set threshold based on the statistical model are designated as the first high-risk locations.

[0021] Based on actual statistics of the locations of violations that have occurred, locations where the frequency of actual violations exceeds a set frequency threshold are designated as the second high-risk locations; the central control unit of the UAV control terminal sets the inspection route of the UAV based on the first and second high-risk locations, and the UAV performs inspections according to the set inspection route.

[0022] Preferably, the construction of the statistical model includes: constructing a mixed location model and a mixed linear model, estimating change points and mixed parameters based on the mixed location model and the mixed linear model, and predicting the location of the violation based on the change points and mixed parameters;

[0023] For mixed data with unknown change points, hypothesis testing is employed, and a test statistic is constructed based on a given empirical process. The existence of a change point is determined based on the different limiting distributions of the test statistic under the null and alternative hypotheses. If the result indicates that a change point exists, for mixed location models with change points, the improved EM algorithm is used to simultaneously estimate the change point location and mixing parameters. The EM algorithm is an iterative algorithm for solving maximum likelihood estimation of parameters from incomplete data or data with missing values.

[0024] The improved EM algorithm includes setting the dataset of already occurred violations as an incomplete dataset, introducing random variables of events that have not yet occurred into the incomplete dataset, taking the log-likelihood function of the dataset with introduced random variables and performing iterative calculation to obtain the mathematical expectation of the latent variables, maximizing the mathematical expectation, obtaining the iterative formula of the mixture parameters based on the maximized mathematical expectation, estimating the change point location and mixture parameters, and obtaining the change point and mixture parameter estimates based on the random variables.

[0025] Mixed linear models are mixed linear models in which the response variable depends on the independent variable. For mixed linear models with change points, the improved EM algorithm is used to estimate the change point location and mixture parameters simultaneously.

[0026] Preferably, the drone control terminal includes a violation database; the violation database records the violation behavior of each violator and the corresponding violation level. The more violations, the higher the violation level; the greater the harm caused by the violation, the higher the violation level; the more frequently violations occur at a violation location, the higher the violation level corresponding to that location and the corresponding violation behavior.

[0027] When the image processing unit determines that there is a violation, it searches the violation database for the corresponding violation level of the violator. The higher the violation level, the higher the warning frequency and the higher the warning volume.

[0028] Preferably, the coverage relationship between the radio frequency reader on the drone and the electronic tag is determined based on the system model;

[0029] A system model is constructed by simulating particle swarm motion. The K-means algorithm is used to adaptively determine the number of wireless RFID readers in the network, and the initial position and power radius of each wireless RFID reader are obtained as a particle. This process is repeated multiple times to obtain a population. Particle swarm algorithm parameters are set to evaluate the performance of various particles in the population and the performance is updated in a hierarchical manner. During the update process of the particle swarm algorithm framework, a virtual force operator is introduced to adjust the position of the wireless RFID reader in each particle.

[0030] Preferably, the wireless RF reader charges the sensor nodes in the wireless network; each agent is simulated as a candidate solution, which can compete with its neighbors, cooperate and share knowledge; a virtual force operator is introduced during the evolution process to attract the wireless RF reader to move towards the node cluster center obtained by the K-means algorithm; and a special crossover operator is set to dynamically adjust the number of wireless RF reader charging locations in the network.

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] This invention provides a field management system based on unmanned aerial vehicle (UAV) automatic inspection, comprising: a UAV, a UAV control terminal, a wireless radio frequency (RFID) reader / writer mounted on the UAV, an electronic tag mounted on a safety helmet or wearable device, and an electronic tag transmission interface. The electronic tag transmission interface is connected to an alarm unit. The UAV control terminal acquires images of the field operation through an image acquisition module and transmits the images to an image processing unit on the UAV control terminal for image processing to determine if there are any violations. If a violation is found, an alarm message is written to the electronic tag on the safety helmet or wearable device of the violator via the RFID reader / writer. The alarm message is transmitted to the alarm unit via the electronic tag transmission interface, and the alarm unit issues an alarm. The UAV control terminal is located at the work site and can analyze and process the behavior of on-site workers. If a violation is found, an alarm is sent to the safety helmet or other wearable device of the violator via the RFID reader / writer on the UAV, providing real-time and effective warnings to the workers involved in the violation. This warning method reduces violations and avoids unnecessary dangers.

[0033] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0034] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0035] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0036] Figure 1 This is a schematic diagram of the structure of a field management system based on unmanned aerial vehicle (UAV) automatic inspection in an embodiment of the present invention;

[0037] Figure 2This is a schematic diagram of the structure of the field management system with a positioning system in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the structure of a field management system with a statistical model in an embodiment of the present invention. Detailed Implementation

[0039] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0040] This invention provides a field management system based on unmanned aerial vehicle (UAV) automated inspection. Please refer to... Figure 1 The system consists of the following parts:

[0041] The drone, the drone control terminal, the wireless radio frequency reader / writer terminal installed on the drone, the electronic tag installed on the safety helmet or wearable device, and the electronic tag transmission interface, wherein the electronic tag transmission interface is connected to the alarm unit;

[0042] The drone control terminal acquires on-site operation images through the image acquisition module and transmits the images to the image processing unit of the drone control terminal for image processing to determine whether there is any violation. If there is a violation, alarm information is written to the electronic tag on the safety helmet or wearable device of the violator through the wireless radio frequency reader / writer. The alarm information is transmitted to the alarm unit through the electronic tag transmission interface, and the alarm unit issues an alarm to the outside.

[0043] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is a drone, a drone control terminal, a wireless radio frequency reader / writer terminal set on the drone, an electronic tag set on a safety helmet or wearable device, and an electronic tag transmission interface, wherein the electronic tag transmission interface is connected to an alarm unit;

[0044] The drone control terminal acquires on-site operation images through the image acquisition module and transmits the images to the image processing unit of the drone control terminal for image processing to determine whether there is any violation. If there is a violation, alarm information is written to the electronic tag on the safety helmet or wearable device of the violator through the wireless radio frequency reader / writer. The alarm information is transmitted to the alarm unit through the electronic tag transmission interface, and the alarm unit issues an alarm to the outside.

[0045] This embodiment uses a wireless radio frequency reader (RFID reader) mounted on a drone and an electronic tag attached to the worker's safety helmet or wearable device. Information is written to the electronic tag via the RFID reader to warn violators. In this embodiment, the RFID reader is not a communication device for controlling the drone's movement or operation, but rather a device for writing information to a designated electronic tag. It can pinpoint a specific violator and alert them via voice broadcast to the safety helmet or wearable device, ensuring that violators are informed of their violations in any complex situation. The alerting effect is improved, and the integration and automation of inspection, violation determination, and warning based on image acquisition and processing modules achieve accurate and effective warnings without adding complexity to the structure.

[0046] The beneficial effects of the above technical solution are as follows: by using the solution provided in this embodiment, the drone control terminal is set up at the work site, which can analyze and process the behavior of on-site workers. If there is any violation, an alarm is sent to the wearable device such as the safety helmet of the violator through the wireless radio frequency reading and writing terminal on the drone. This can provide real-time and effective warnings to the workers corresponding to the violations, thereby reducing violations and avoiding unnecessary dangers.

[0047] In another embodiment, if the image processing unit determines that the violation is the act of not wearing a safety helmet, the wireless radio frequency reader / writer obtains the location of the safety helmet corresponding to the violator by reading the electronic tag through wireless radio frequency technology; the electronic tag is equipped with a GPS positioning module for generating the location information of the electronic tag; the wireless radio frequency reader / writer reads the location information when reading the electronic tag information;

[0048] Once the location information of the safety helmet of the violator is obtained, the wireless radio frequency reader / writer writes alarm information and location information to the electronic tag on the wearable device corresponding to the violator; the alarm information and location information are transmitted to the alarm unit through the electronic tag transmission interface, and when the alarm unit sends out an alarm, it broadcasts the location information of the safety helmet.

[0049] The working principle of the above technical solution is as follows: In this embodiment, if the image processing unit determines that the violation is a violation of not wearing a safety helmet, the wireless radio frequency reader / writer obtains the location of the safety helmet corresponding to the violator by reading the electronic tag through wireless radio frequency technology; the electronic tag is equipped with a GPS positioning module to generate the location information of the electronic tag; the wireless radio frequency reader / writer reads the location information when reading the electronic tag information.

[0050] Once the location information of the safety helmet of the violator is obtained, the wireless radio frequency reader / writer writes alarm information and location information to the electronic tag on the wearable device corresponding to the violator; the alarm information and location information are transmitted to the alarm unit through the electronic tag transmission interface, and when the alarm unit sends out an alarm, it broadcasts the location information of the safety helmet.

[0051] The beneficial effects of the above technical solution are as follows: Using the positioning module provided in this embodiment, the location of the safety helmet can be further determined, reminding violators of its location and facilitating their quickest access to the helmet and correction of their violation. Simultaneously, it can prevent the helmet from being lost.

[0052] In another embodiment, the drone control terminal is equipped with a personnel information database. In this personnel information database, the identity information of the same personnel corresponds to two different electronic tag identifiers, and each electronic tag has a unique electronic tag identifier.

[0053] An electronic tag is set on both the safety helmet and the wearable device of the same worker; if the worker changes the safety helmet or wearable device, the updated electronic tag is displayed in the worker's information database.

[0054] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is that the drone control terminal is equipped with a personnel information database. In this personnel information database, the identity information of the same personnel corresponds to two different electronic tag identifiers, and each electronic tag has a unique electronic tag identifier.

[0055] An electronic tag is set on both the safety helmet and the wearable device of the same worker; if the worker changes the safety helmet or wearable device, the updated electronic tag is displayed in the worker's information database.

[0056] The beneficial effects of the above technical solution are as follows: the solution provided in this embodiment can update the correspondence between electronic tags and operators in real time through the database. The database records avoid the situation where the electronic tag that triggers the alarm is not the electronic tag corresponding to the violator due to untimely information updates. Therefore, this embodiment can ensure information synchronization and improve efficiency.

[0057] In another embodiment, the image acquisition module includes an image acquisition device mounted on the drone, a directional camera or a mobile camera mounted at the work site;

[0058] During the inspection process, the drone sends its current location to the drone control terminal. The drone control terminal then sends acquisition and transmission commands to image acquisition modules within a preset range of the drone's location via wireless communication. The image acquisition modules within the preset range acquire image information within the current time period and transmit the image information to the drone control terminal.

[0059] The image processing unit of the UAV control terminal filters the received image information, filters out images with workers, and analyzes the violations in the filtered images.

[0060] During the analysis of violations, feature information of the image to be analyzed is extracted and matched with features in the violation database. If one or more matching features are found, the violation corresponding to that feature is marked. Each violation corresponds to a standard behavior. The text information of the violation and the standard behavior is transmitted to the alarm unit. The alarm unit converts the text information into voice information and issues a voice alarm. The standard behavior reminds the operator to correct the violation.

[0061] In the process of extracting feature information from the image to be analyzed, a bilinear convolutional neural network is constructed based on the prototype network as the network framework and the MobileNet-V2 and ResNet-18 networks as the embedding function of the prototype network to perform feature recognition on the image to be analyzed and obtain feature information.

[0062] In the process of feature recognition, sample features are analyzed using the training set, and the sample data is mapped to a multi-dimensional space using an embedding function. Based on the embedding function, the loss function of the prototype network is determined. The loss function is as follows:

[0063]

[0064] Where L′ represents the updated loss function, and L represents the loss function. Indicates training set sample x i To category prototype c k The Euclidean distance, c k x represents the prototype of category k. i Represents training set samples, Indicates training set sample x i The corresponding embedding function, c j This represents the prototype of category j, where j = 1, 2, ..., N, and N represents the number of category prototypes. This represents the sum of distances between training set samples and N class prototypes, where N is the total distance. p N represents the number of support set samples for each class in the training set. q This represents the number of samples for each type of query set in the training set.

[0065] The working principle of the above technical solution is as follows: if reminding the violator is ineffective, and if the behavior is still not corrected after 10 seconds or 1 minute, i.e., the violation is still detected upon re-detection, then the system uses facial recognition to send reminder instructions to the workers around the violator. This allows the workers to remind the violator in real time, improving the efficiency and effectiveness of timely reminders.

[0066] Furthermore, in the process of analyzing traffic violations, feature information of the image to be analyzed is extracted. Prototype networks are used for feature extraction, a common method for handling few-shot learning tasks. This method is a metric-based meta-learning approach that is highly effective in solving the problem of numerous image categories and a small number of labeled samples in image recognition tasks. The execution steps of a prototype network are as follows: Each category of the sample data has a prototype representation. First, the sample data is mapped to a new embedding space using an embedding function. By calculating the mean of all values ​​of the labeled samples in the support set of the test set in the mapping space, the prototype of each category is obtained. Finally, the Euclidean distance from the feature vectors of the query set samples in the training set to the prototypes of each category is calculated. The recognition and classification task is completed through a similarity metric. Therefore, prototype networks transform a complex classification problem into a nearest neighbor problem of samples in the embedding space, enabling them to achieve high recognition accuracy in few-shot recognition tasks on fine-grained image datasets.

[0067] The beneficial effects of the above technical solution are as follows: This embodiment adopts a network architecture based on a prototype network. The backbone network selects MobileNet-V2 and ResNet-18 networks to construct a bilinear convolutional neural network, realizing the recognition task of small sample fine-grained images. This enhances the learning ability of the computer.

[0068] In another embodiment, the drone control terminal is wirelessly connected to the image acquisition module, and there are multiple drone control terminals set up at the work site;

[0069] The drone control terminal is equipped with a timing unit. When the drone control terminal controls the alarm unit to issue a warning via the wireless radio frequency reader / writer, the timing unit is activated. After a preset time, the drone control terminal sends a command to the image acquisition module to re-acquire the image information of the violator. If the image analysis unit determines that the violator still has a violation, the image acquisition module acquires the facial information of the workers around the violator, and the facial recognition module identifies the identity information of the corresponding workers. Based on the identity information, the corresponding electronic tag of the worker is obtained, and a reminder command is written to the corresponding electronic tag via the wireless radio frequency reader / writer on the drone. The reminder command includes the violator's name, employee number, violation, and compliant behavior. The alarm unit on the safety helmet or wearable device of the reminded worker sounds an alarm and broadcasts the reminder command via voice, so that the reminded worker can help the violator correct the violation in a timely manner.

[0070] The working principle of the above technical solution is as follows: In this embodiment, the UAV control terminal and the image acquisition module are connected wirelessly. There are multiple UAV control terminals, which are set up at the work site.

[0071] The drone control terminal is equipped with a timing unit. When the drone control terminal controls the alarm unit to issue a warning via the wireless radio frequency reader / writer, the timing unit is activated. After a preset time, the drone control terminal sends a command to the image acquisition module to re-acquire the image information of the violator. If the image analysis unit determines that the violator still has a violation, the image acquisition module acquires the facial information of the workers around the violator, and the facial recognition module identifies the identity information of the corresponding workers. Based on the identity information, the corresponding electronic tag of the worker is obtained, and a reminder command is written to the corresponding electronic tag via the wireless radio frequency reader / writer on the drone. The reminder command includes the violator's name, employee number, violation, and compliant behavior. The alarm unit on the safety helmet or wearable device of the reminded worker sounds an alarm and broadcasts the reminder command via voice, so that the reminded worker can help the violator correct the violation in a timely manner.

[0072] The beneficial effects of the above technical solution are as follows: When the person violating the rules does not perform the prescribed behavior after being warned, for example, 30 seconds or one minute after the alarm, and the person is still in a violating state when the violation is detected, the solution provided in this embodiment will remind the workers around the person violating the rules through facial recognition, so that the workers around the person violating the rules can remind the person violating the rules, reduce violations, and improve the efficiency and effectiveness of violation alarms.

[0073] In another embodiment, the drone control terminal further includes a statistical model;

[0074] Locations where the probability of a violation is predicted to be higher than a set threshold based on the statistical model are designated as the first high-risk locations.

[0075] Based on actual statistics of the locations of violations that have occurred, locations where the frequency of actual violations exceeds a set frequency threshold are designated as the second high-risk locations; the central control unit of the UAV control terminal sets the inspection route of the UAV based on the first and second high-risk locations, and the UAV performs inspections according to the set inspection route.

[0076] The working principle of the above technical solution is as follows: the solution adopted in this embodiment is that the UAV control terminal also includes a statistical model;

[0077] Locations where the probability of a violation is predicted to be higher than a set threshold based on the statistical model are designated as the first high-risk locations.

[0078] Based on actual statistics of the locations of violations that have occurred, locations where the frequency of actual violations exceeds a set frequency threshold are designated as the second high-risk locations; the central control unit of the UAV control terminal sets the inspection route of the UAV based on the first and second high-risk locations, and the UAV performs inspections according to the set inspection route.

[0079] The beneficial effects of the above technical solution are as follows: the UAV control terminal using the solution provided in this embodiment also includes a statistical model;

[0080] Locations where the probability of a violation is predicted to be higher than a set threshold based on the statistical model are designated as the first high-risk locations.

[0081] Based on actual statistics of the locations of violations that have occurred, locations where the frequency of actual violations exceeds a set frequency threshold are designated as the second high-risk locations; the central control unit of the UAV control terminal sets the inspection route of the UAV based on the first and second high-risk locations, and the UAV performs inspections according to the set inspection route.

[0082] In another embodiment, the construction of the statistical model includes: constructing a mixed location model and a mixed linear model, estimating change points and mixed parameters based on the mixed location model and the mixed linear model, and predicting the location of the violation based on the change points and mixed parameters;

[0083] For mixed data with unknown change points, hypothesis testing is employed, and a test statistic is constructed based on a given empirical process. The existence of a change point is determined based on the different limiting distributions of the test statistic under the null and alternative hypotheses. If the result indicates that a change point exists, for mixed location models with change points, the improved EM algorithm is used to simultaneously estimate the change point location and mixing parameters. The EM algorithm is an iterative algorithm for solving maximum likelihood estimation of parameters from incomplete data or data with missing values.

[0084] The improved EM algorithm includes setting the dataset of already occurred violations as an incomplete dataset, introducing random variables of events that have not yet occurred into the incomplete dataset, taking the log-likelihood function of the dataset with introduced random variables and performing iterative calculation to obtain the mathematical expectation of the latent variables, maximizing the mathematical expectation, obtaining the iterative formula of the mixture parameters based on the maximized mathematical expectation, estimating the change point location and mixture parameters, and obtaining the change point and mixture parameter estimates based on the random variables.

[0085] Mixed linear models are mixed linear models in which the response variable depends on the independent variable. For mixed linear models with change points, the improved EM algorithm is used to estimate the change point location and mixture parameters simultaneously.

[0086] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is to construct a hybrid location model and a hybrid linear model, estimate the change point and hybrid parameters based on the hybrid location model and the hybrid linear model, and predict the location of the violation based on the change point and hybrid parameters; the change point location and hybrid parameters are estimated simultaneously through an improved EM algorithm; the improved EM algorithm includes setting the dataset of the violations that have occurred as an incomplete dataset, introducing random variables of events that have not occurred into the incomplete dataset, taking the log-likelihood function of the dataset with introduced random variables, and performing iterative calculation to obtain the mathematical expectation of the latent variables, maximizing the mathematical expectation, obtaining the iterative formula of the hybrid parameters based on the maximized mathematical expectation, estimating the change point location and hybrid parameters, and obtaining the estimates of the change point and hybrid parameters based on the random variables.

[0087] The beneficial effects of the above technical solution are as follows: the solution provided in this embodiment solves the problem of parameter estimation for mixed data. The general solution is to ignore the changing points and directly use the EM algorithm to estimate the parameters. Furthermore, the changing point estimator and the mixed parameter estimator are consistent.

[0088] In another embodiment, the coverage relationship between the radio frequency reader on the drone and the electronic tag is determined based on the system model;

[0089] A system model is constructed by simulating particle swarm motion. The K-means algorithm is used to adaptively determine the number of wireless RFID readers in the network, and the initial position and power radius of each wireless RFID reader are obtained as a particle. This process is repeated multiple times to obtain a population. Particle swarm algorithm parameters are set to evaluate the performance of various particles in the population and the performance is updated in a hierarchical manner. During the update process of the particle swarm algorithm framework, a virtual force operator is introduced to adjust the position of the wireless RFID reader in each particle.

[0090] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is the coverage relationship between the radio frequency reader and the electronic tag on the drone, which is determined based on the system model;

[0091] A system model is constructed by simulating particle swarm motion. The K-means algorithm is used to adaptively determine the number of wireless RFID readers in the network, and the initial position and power radius of each wireless RFID reader are obtained as a particle. This process is repeated multiple times to obtain a population. Particle swarm algorithm parameters are set to evaluate the performance of various particles in the population and the performance is updated in a hierarchical manner. During the update process of the particle swarm algorithm framework, a virtual force operator is introduced to adjust the position of the wireless RFID reader in each particle.

[0092] The beneficial effects of the above technical solution are as follows: the hybrid particle swarm algorithm proposed based on the network planning problem can effectively determine the number and location of wireless radio frequency readers, making the RFID system outperform other algorithms in terms of interference rate, coverage, total power and load balance.

[0093] In another embodiment, the radio frequency reader charges the sensor nodes in the wireless network; each agent is simulated as a candidate solution that can compete, cooperate and share knowledge with its neighbors; a virtual force operator is introduced during the evolution process to attract the radio frequency reader to move toward the node cluster center obtained by the K-means algorithm; and a special crossover operator is set to dynamically adjust the number of charging locations for the radio frequency reader in the network.

[0094] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is that the wireless radio frequency reader charges the sensor nodes in the wireless network; each agent is simulated as a candidate solution, which can compete with its neighbors, cooperate and share knowledge; a virtual force operator is introduced in the evolution process to attract the wireless radio frequency reader to move towards the node cluster center obtained by the K-means algorithm; a special crossover operator is set to dynamically adjust the number of wireless radio frequency reader charging positions in the network.

[0095] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, the algorithm can more effectively optimize the total charging time, maximum charging load, charging efficiency and total charging distance.

[0096] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A field management system based on unmanned aerial vehicle (UAV) automatic inspection, characterized in that, include: The drone, the drone control terminal, the wireless radio frequency reader / writer terminal installed on the drone, the electronic tag installed on the safety helmet or wearable device, and the electronic tag transmission interface, wherein the electronic tag transmission interface is connected to the alarm unit; The drone control terminal acquires on-site operation images through the image acquisition module and transmits the images to the image processing unit of the drone control terminal for image processing to determine whether there is any violation. If there is a violation, alarm information is written to the electronic tag on the safety helmet or wearable device of the violator through the wireless radio frequency reader / writer terminal. The alarm information is transmitted to the alarm unit through the electronic tag transmission interface and the alarm unit sends out an alarm. The image acquisition module includes an image acquisition device mounted on the drone, or a directional camera or a mobile camera set up at the work site; During the inspection process, the drone sends its current location to the drone control terminal. The drone control terminal then sends acquisition and transmission commands to image acquisition modules within a preset range of the drone's location via wireless communication. The image acquisition modules within the preset range acquire image information within the current time period and transmit the image information to the drone control terminal. The image processing unit of the UAV control terminal filters the received image information, filters out images with workers, and analyzes the violations in the filtered images. During the analysis of violations, feature information of the image to be analyzed is extracted and matched with features in the violation database. If one or more matching features are found, the violation corresponding to that feature is marked. Each violation corresponds to a standard behavior. The text information of the violation and the standard behavior is transmitted to the alarm unit. The alarm unit converts the text information into voice information and issues a voice alarm. The standard behavior reminds the operator to correct the violation. In the process of extracting feature information from the image to be analyzed, a bilinear convolutional neural network is constructed based on the prototype network as the network framework and the MobileNet-V2 network and the ResNet-18 network as the embedding function of the prototype network to perform feature recognition on the image to be analyzed and obtain feature information. If the image processing unit determines that the violation is due to not wearing a safety helmet, the wireless radio frequency reader / writer obtains the location of the safety helmet corresponding to the violator by reading the electronic tag through wireless radio frequency technology; the electronic tag is equipped with a GPS positioning module to generate the location information of the electronic tag; the wireless radio frequency reader / writer reads the location information when reading the electronic tag information. Once the location information of the safety helmet of the violator is obtained, the wireless radio frequency reader / writer writes alarm information and location information to the electronic tag on the wearable device corresponding to the violator; the alarm information and location information are transmitted to the alarm unit through the electronic tag transmission interface, and when the alarm unit sends out an alarm, it broadcasts the location information of the safety helmet.

2. The on-site management system based on unmanned aerial vehicle (UAV) automatic inspection according to claim 1, characterized in that, The drone control terminal is equipped with a personnel information database. In this personnel information database, the identity information of the same operator corresponds to two different electronic tag identifiers, and each electronic tag has a unique electronic tag identifier. An electronic tag is set on both the safety helmet and the wearable device of the same worker; if the worker changes the safety helmet or wearable device, the updated electronic tag is displayed in the worker's information database.

3. The on-site management system based on unmanned aerial vehicle (UAV) automatic inspection according to claim 1, characterized in that, The drone control terminal and the image acquisition module are connected wirelessly. There are multiple drone control terminals, which are set up at the work site. The drone control terminal is equipped with a timing unit. When the drone control terminal controls the alarm unit to send an alarm through the wireless radio frequency reader / writer, the timing unit is activated. After a preset time, the drone control terminal sends a command to the image acquisition module to re-acquire the image information of the violator. If the image processing unit determines through analysis that the violator still has a violation, the image acquisition module acquires the facial information of the workers around the violator, and the facial recognition module identifies the identity information of the corresponding workers. Based on the identity information, the corresponding electronic tag of the worker is obtained, and a reminder command is written to the electronic tag of the worker through the wireless radio frequency reader / writer on the drone. The reminder command includes the violator's name, employee number, violation, and compliant behavior. The alarm unit on the safety helmet or wearable device of the reminded worker sends an alarm and broadcasts the reminder command via voice, so that the reminded worker can help the violator correct the violation in a timely manner.

4. The on-site management system based on unmanned aerial vehicle (UAV) automatic inspection according to claim 1, characterized in that, The UAV control terminal also includes a statistical model; Locations where the probability of a violation is predicted to be higher than a set threshold based on the statistical model are designated as the first high-risk locations. The locations of the violations that have occurred are statistically analyzed, and the locations where the frequency of actual violations exceeds a set frequency threshold are set as the second high-risk locations; the central control unit of the UAV control terminal sets the inspection route of the UAV based on the first and second high-risk locations, and the UAV performs inspections according to the set inspection route.

5. A field management system based on unmanned aerial vehicle (UAV) automatic inspection according to claim 4, characterized in that, The construction of the statistical model includes: constructing a mixed location model and a mixed linear model, estimating change points and mixed parameters based on the mixed location model and the mixed linear model, and predicting the location of the violation based on the change points and mixed parameters; For mixed data with unknown change points, hypothesis testing is employed, and a test statistic is constructed based on a given empirical process. The existence of a change point is determined based on the different limiting distributions of the test statistic under the null and alternative hypotheses. If the result indicates that a change point exists, for mixed location models with change points, the improved EM algorithm is used to simultaneously estimate the change point location and mixing parameters. The EM algorithm is an iterative algorithm for solving maximum likelihood estimation of parameters from incomplete data or data with missing values. The improved EM algorithm includes setting the dataset of already occurred violations as an incomplete dataset, introducing random variables of events that have not yet occurred into the incomplete dataset, taking the log-likelihood function of the dataset with introduced random variables and performing iterative calculation to obtain the mathematical expectation of the latent variables, maximizing the mathematical expectation, obtaining the iterative formula of the mixture parameters based on the maximized mathematical expectation, estimating the change point location and mixture parameters, and obtaining the change point and mixture parameter estimates based on the random variables. Mixed linear models are mixed linear models in which the response variable depends on the independent variable. For mixed linear models with change points, the improved EM algorithm is used to estimate the change point location and mixture parameters simultaneously.

6. A field management system based on unmanned aerial vehicle (UAV) automatic inspection according to claim 1, characterized in that, The drone control terminal includes a violation database; the violation database records the violation behavior of each violator and the corresponding violation level. The more violations, the higher the violation level; the greater the harm caused by the violation, the higher the violation level. The more frequently violations occur at a location, the higher the violation level of the corresponding violations at that location. When the image processing unit determines that there is a violation, it searches the violation database for the corresponding violation level of the violator. The higher the violation level, the higher the warning frequency and the higher the warning volume.

Images