Method for detecting railway operation behavior based on image

Abstract

The invention relates to a method for detecting railway operation behaviors based on images, and relates to the field of image processing.The method comprises the steps that a real-time video of a target object in a preset area in the moving process, the position of the target object in a train and a to-be-detected item closest to the position are obtained, and when the position reaches the position corresponding to the to-be-detected item, the to-be-detected item is detected; and outputting prompt information, analyzing the real-time video to judge whether the target object accords with a preset acquisition attitude of the to-be-detected item, if so, acquiring image information of the to-be-detected item, and if not, determining the data acquisition quality of the target object to the to-be-detected item based on the image information. And if the data acquisition quality is lower than the preset quality threshold, controlling the image acquisition device to reacquire the image information of the to-be-detected item. According to the invention, the efficiency of inspection and photographing of mechanicians can be improved when poor quality such as picture blurring occurs, and photographing omission is not easy to occur.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for detecting railway operation behavior based on images. Background Art

[0002] Railways are one of the main means of transportation for people to travel. Safety inspections, emergency response and daily maintenance are the top priorities of railway operation and management. To ensure the safe operation of railway lines, each train is usually equipped with a mechanic. With the extension of the line and the improvement of train intelligence, the number of equipment and links that mechanics need to monitor has also increased accordingly. Mechanics need to take photos of trains for inspection, but due to the long trains and complex internal environments, mechanics may miss photos at certain times or in certain areas, affecting a comprehensive understanding of the train status and troubleshooting. In addition, when the train is traveling at high speed, factors such as body vibration and light changes may cause blurry photos, or poor shooting angles, making it difficult to identify key information. Missed photos and blurred photos need to be re-photographed afterwards, resulting in reduced inspection efficiency. Summary of the invention

[0003] In order to improve the efficiency of mechanics' inspection photography and reduce the risk of missing photos when photos are blurry or of poor quality, the present application provides an image-based method for detecting railway operation behaviors.

[0004] In a first aspect, the present application provides a method for detecting railway operation behavior based on an image, which adopts the following technical solution: A method for detecting railway operation behavior based on an image, comprising: Acquire a real-time video of a target object in a preset area during its activity, the position of the target object in the train, and an item to be detected closest to the position, wherein the real-time video is acquired by a video acquisition device on the target object; When the position reaches the position corresponding to the item to be detected, a prompt message is output, and the real-time video is analyzed to determine whether the target object meets the preset collection posture of the item to be detected; If the preset acquisition posture is met, image information of the item to be detected is acquired, where the image information is acquired by the target object through an image acquisition device; Determining the data acquisition quality of the target object for the item to be detected based on the image information; If the data acquisition quality is lower than a preset quality threshold, the image acquisition device is controlled to re-acquire image information of the item to be detected.

[0005] By adopting the above technical solution, the real-time video, position and the nearest item to be detected during the activity of the target object are obtained. When it is determined that the target object has arrived at the position of the item to be detected, a prompt message is output to remind the target object to arrive at the item to be detected, so that the target object is not likely to miss or miss the item to be detected. Since the real-time video is collected by the video acquisition device on the target object, the viewing angle of the real-time video changes with the movement of the target object. According to the real-time video, it can be judged whether the target object meets the corresponding posture when collecting image information of the item to be detected, that is, the preset acquisition posture. If it meets the preset acquisition posture, it means that the next image acquisition operation can be carried out to obtain the image information collected by the image acquisition device of the target object, and the data acquisition quality of the target object is determined according to the image information. If the data acquisition device is lower than the preset quality threshold, it means that the display effect of the item to be detected in the image information is poor, and the specific situation of the item to be detected cannot be known. Therefore, the image acquisition device is re-controlled to collect the image information of the item to be detected, which is more efficient than re-shooting afterwards when the photo is blurred.

[0006] In another possible implementation, the analyzing the real-time video to determine whether the target object meets the preset collection posture of the item to be detected includes: Performing feature recognition on the real-time video to determine whether a first identifier is recognized; If the first marker is identified, determining the position and outline of the first marker in the picture of the real-time video; If the first mark is located in a preset area in the picture, determining the area of ​​the outline; A first ratio of the area to a preset area threshold is determined, and if the first ratio is within a preset ratio interval, it is determined that the target object has reached a preset collection posture of the item to be detected.

[0007] In another possible implementation manner, determining the data acquisition quality of the target object for the item to be detected based on the image information includes: Performing feature recognition on the image information to determine whether a second mark exists in the image information, the second mark being set in a preset area of ​​the item to be detected, and the number of the second marks being multiple, each of which has a different shape; If the second marker exists, determining a first number of the second markers in the image information, and performing edge detection on the image information to obtain a contour of each second marker and a sub-image of each second marker; Determine the similarity between the outline of each second mark and the corresponding standard outline, and determine the definition difference between the sub-image of each second mark and the corresponding standard image; Determining whether each second mark is located in a corresponding preset area in the picture; If there is a target second identifier located in the corresponding preset area, determining a second number of target second identifiers; The data acquisition quality is determined based on the first number, the second number, the similarity, and the clarity difference.

[0008] In another possible implementation, determining the data acquisition quality based on the first quantity, the second quantity, the similarity, and the clarity difference includes: Determine a second ratio of the first quantity to the quantity of second identifiers corresponding to the items to be detected; Determining a third ratio of the second quantity to the first quantity; Determining a first average value of the similarities and a second average value of the clarity differences; A first score is determined based on the second proportion, the third proportion, the first average value, and the second average value, where the first score represents the data collection quality of the target object on the item to be detected.

[0009] In another possible implementation, the controlling the image acquisition device to re-acquire image information of the item to be detected includes: Controlling the image acquisition device to display the preset area of ​​each second mark, and detecting in real time whether each second mark is located in the corresponding preset area; When it is detected that each second identification is located in the corresponding preset area and the time for which they are located in the corresponding preset area reaches a preset time, the image acquisition device is controlled to acquire a preset number of image information of the item to be detected, and the preset number is determined based on the similarity and clarity difference.

[0010] In another possible implementation manner, determining the preset number includes: Determine a third number of first target second identifiers and an average value of similarities corresponding to the first target second identifiers, wherein the first target second identifiers include second identifiers whose similarity is lower than a preset similarity threshold; Determine a fourth number of second target second identifiers and an average value of clarity differences corresponding to the second target second identifiers, wherein the second target second identifiers include second identifiers whose clarity differences are lower than a preset clarity difference threshold; Determine a fourth ratio of the average similarity value to the third number and a fifth ratio of the average clarity difference value to the fourth number; Determine a second score based on the fourth proportion, the fifth proportion and their respective corresponding coefficients; The preset number is determined based on the second score.

[0011] In another possible implementation manner, if the data collection quality reaches a preset quality threshold, the method further includes: Preprocessing the image information to obtain preprocessed image information; Performing element detection on the preprocessed image information to obtain target elements; Performing text recognition on the preprocessed image information to obtain text information; Based on the text information and the target element, it is determined whether the item to be detected corresponding to the image information is abnormal.

[0012] In a second aspect, the present application provides an image-based device for detecting railway operation behavior, which adopts the following technical solution: A device for detecting railway operation behavior based on an image, comprising: A data acquisition module is used to acquire real-time video of a target object in a preset area during its activity, the position of the target object in the train, and the nearest item to be detected, wherein the real-time video is acquired by a video acquisition device on the target object; A judgment module, configured to output a prompt message when the position reaches the position corresponding to the item to be detected, and analyze the real-time video to determine whether the target object meets the preset collection posture of the item to be detected; An image acquisition module, used for acquiring image information of the item to be detected if the preset acquisition posture is met, wherein the image information is acquired by the target object through an image acquisition device; A quality determination module, used to determine the data acquisition quality of the target object for the item to be detected based on the image information; The control module is used to control the image acquisition device to re-acquire the image information of the item to be detected when the data acquisition quality is lower than a preset quality threshold.

[0013] By adopting the above technical solution, the data acquisition module obtains the real-time video, position and the nearest item to be detected during the activity of the target object. When it is determined that the target object has arrived at the position of the item to be detected, the judgment module outputs a prompt message to remind the target object to arrive at the item to be detected, so that the target object is not easy to miss or miss the item to be detected. Since the real-time video is collected by the video acquisition device on the target object, the viewing angle of the real-time video changes with the movement of the target object. According to the real-time video of the judgment module, it can be judged whether the target object meets the corresponding posture when collecting image information of the item to be detected, that is, the preset collection posture. If it meets the preset collection posture, it means that the next step of image collection operation can be carried out. The image acquisition module obtains the image information collected by the target object through the image acquisition device. The quality determination module determines the data collection quality of the target object according to the image information. If the data acquisition device is lower than the preset quality threshold, it means that the display effect of the item to be detected in the image information is poor, and the specific situation of the item to be detected cannot be known. Therefore, the control module re-controls the image acquisition device to collect the image information of the item to be detected, which is more efficient than re-shooting after the photo is blurred.

[0014] In another possible implementation, when the judgment module analyzes the real-time video to determine whether the target object meets the preset collection posture of the item to be detected, it is specifically used to: Performing feature recognition on the real-time video to determine whether a first identifier is recognized; If the first marker is identified, determining the position and outline of the first marker in the picture of the real-time video; If the first mark is located in a preset area in the picture, determining the area of ​​the outline; A first ratio of the area to a preset area threshold is determined, and if the first ratio is within a preset ratio interval, it is determined that the target object has reached a preset collection posture of the item to be detected.

[0015] In another possible implementation, when determining the data acquisition quality of the target object for the item to be detected based on the image information, the quality determination module is specifically configured to: Performing feature recognition on the image information to determine whether a second mark exists in the image information, the second mark being set in a preset area of ​​the item to be detected, and the number of the second marks being multiple, each of which has a different shape; If the second marker exists, determining a first number of the second markers in the image information, and performing edge detection on the image information to obtain a contour of each second marker and a sub-image of each second marker; Determine the similarity between the outline of each second mark and the corresponding standard outline, and determine the definition difference between the sub-image of each second mark and the corresponding standard image; Determining whether each second mark is located in a corresponding preset area in the picture; If there is a target second identifier located in the corresponding preset area, determining a second number of target second identifiers; The data acquisition quality is determined based on the first number, the second number, the similarity, and the clarity difference.

[0016] In another possible implementation, when the quality determination module determines the data acquisition quality based on the first quantity, the second quantity, the similarity, and the clarity difference, it is specifically configured to: Determine a second ratio of the first quantity to the quantity of second identifiers corresponding to the items to be detected; Determining a third ratio of the second quantity to the first quantity; Determining a first average value of the similarities and a second average value of the clarity differences; A first score is determined based on the second proportion, the third proportion, the first average value, and the second average value, where the first score represents the data collection quality of the target object on the item to be detected.

[0017] In another possible implementation, when the control module controls the image acquisition device to re-acquire the image information of the item to be detected, it is specifically used to: Controlling the image acquisition device to display the preset area of ​​each second mark, and detecting in real time whether each second mark is located in the corresponding preset area; When it is detected that each second identification is located in the corresponding preset area and the time for which they are located in the corresponding preset area reaches a preset time, the image acquisition device is controlled to acquire a preset number of image information of the item to be detected, and the preset number is determined based on the similarity and clarity difference.

[0018] In another possible implementation, the determining the preset number, the device further includes: A first determination module, configured to determine a third number of first target second identifiers and an average similarity value corresponding to the first target second identifiers, wherein the first target second identifiers include second identifiers whose similarity is lower than a preset similarity threshold; A second determination module is used to determine a fourth number of second target second identifiers and an average value of clarity differences corresponding to the second target second identifiers, wherein the second target second identifiers include second identifiers whose clarity differences are lower than a preset clarity difference threshold; A third determination module is used to determine a fourth ratio of the average similarity value to the third number and a fifth ratio of the average clarity difference value to the fourth number; A fourth determination module, configured to determine a second score based on the fourth proportion, the fifth proportion and their respective corresponding coefficients; A fifth determining module is configured to determine the preset number based on the second score.

[0019] In another possible implementation, the device further includes: A preprocessing module, used for preprocessing the image information to obtain preprocessed image information; An element detection module, used to perform element detection on the preprocessed image information to obtain a target element; A text recognition module, used to perform text recognition on the pre-processed image information to obtain text information; The abnormality judgment module is used to judge whether the to-be-detected item corresponding to the image information is abnormal based on the text information and the target element.

[0020] In a third aspect, the present application provides an electronic device, which adopts the following technical solution: An electronic device, comprising: at least one processor; Memory; At least one application, wherein at least one application is stored in a memory and configured to be executed by at least one processor, and at least one is configured to: execute an image-based method for detecting railway operation behavior as shown in any possible implementation of the first aspect.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium, when the computer program is executed in a computer, causes the computer to execute the method for detecting railway operation behavior based on an image as described in any one of the first aspects.

[0022] In summary, the present application includes at least one of the following beneficial technical effects: The real-time video, position and the nearest item to be detected during the activity of the target object are obtained. When it is determined that the target object has arrived at the position of the item to be detected, a prompt message is output to remind the target object to arrive at the item to be detected, so that the target object is not likely to miss or miss the item to be detected. Since the real-time video is collected by the video acquisition device on the target object, the viewing angle of the real-time video changes with the movement of the target object. According to the real-time video, it can be judged whether the target object meets the corresponding posture when collecting image information of the item to be detected, that is, the preset acquisition posture. If it meets the preset acquisition posture, it means that the next image acquisition operation can be carried out to obtain the image information collected by the image acquisition device of the target object, and the data acquisition quality of the target object is determined according to the image information. If the image acquisition quality is lower than the preset quality threshold, it means that the display effect of the item to be detected in the image information is poor, and the specific situation of the item to be detected cannot be known. Therefore, the image acquisition device is re-controlled to collect the image information of the item to be detected, which is more efficient than re-shooting afterwards when the photo is blurred. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flowchart of a method for detecting railway operation behavior based on an image according to an embodiment of the present application.

[0024] Figure 2 It is a structural schematic diagram of a device for detecting railway operation behavior based on an image according to an embodiment of the present application.

[0025] Figure 3 It is a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The present application is further described in detail below in conjunction with the accompanying drawings.

[0027] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but such modifications are protected by patent law as long as they are within the scope of the claims of this application.

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0030] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0031] The embodiment of the present application provides a method for detecting railway operation behavior based on an image, which is executed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and the embodiment of the present application does not limit this. Figure 1 As shown, the method includes step S101, step S102, step S103, step S104 and step S105, wherein: S101, obtaining a real-time video of a target object in a preset area during its activity, the position of the target object in the train, and the nearest item to be detected.

[0032] The real-time video is collected by a video collection device on the target object.

[0033] For the embodiment of the present application, the target object is a mechanic who is inspecting in the train. The mechanic carries a positioning module, so the positioning in the train can be achieved through Ultra Wide Band (UWB) technology; the position can also be determined through WIFI positioning, and the position is estimated by receiving the strength of the Wi-Fi signal and the signal arrival time difference; Bluetooth can also be used for positioning, which is not limited here. Taking UWB technology as an example, the electronic device and the positioning module are connected and communicated wirelessly. The staff needs to deploy UWB anchor points (Anchor Points) at known locations in the train. The positioning module, that is, the tag (Tags), is attached to the object or person to be located. The distance between the anchor point and the tag can be determined by measuring the transmission time (Time of Flight, TOF) of the signal. The position of the tag is calculated using trilateral measurement or multilateral measurement algorithms. These algorithms need to know the position information of the anchor point. By measuring the distance from the tag to multiple anchor points, the precise position of the tag can be calculated using the triangulation positioning principle, so that the electronic device can obtain the position of the mechanic in the train.

[0034] The target object, i.e., the mechanic, is wearing a video acquisition device, such as a recorder, a sports camera, etc. The preset area is the area where the video acquisition device collects video. The video acquisition device is connected to the electronic device wirelessly, and the electronic device obtains real-time video, so as to facilitate the recording of the mechanic's inspection process. In addition, since different inspection items are located in different positions in the train, the mechanic needs to adjust his posture after arriving at the position corresponding to the inspection item in order to more conveniently collect images. Since the video acquisition device is set on the mechanic, the video acquisition device changes position following the mechanic's movements. Therefore, it is convenient to determine the mechanic's posture when collecting images for the inspection items based on the real-time video.

[0035] The electronic device stores the location of each item to be inspected on the train. Therefore, after the electronic device obtains the location of the mechanic, it can calculate the distance between the mechanic's location and the location of each item to be inspected, thereby determining the closest item to be inspected. Since the mechanic usually conducts inspections in the order of inspection items, the closest item to be inspected is usually the next inspection item in the order. The mechanic collects image information of the item to be inspected, thereby completing the image acquisition operation.

[0036] S102, when the position reaches the position corresponding to the item to be detected, output prompt information, and analyze the real-time video to determine whether the target object meets the preset collection posture of the item to be detected.

[0037] For the embodiment of the present application, the location of the item to be detected on the train can be indicated by the ground or operating surface near the item to be detected. When the mechanic reaches the ground or operating surface near the item to be detected, the electronic device determines that the mechanic has reached the location corresponding to the item to be detected. The prompt information can be that the electronic device can send a prompt instruction to the positioning device, i.e., the tag, on the mechanic. After the positioning device, i.e., the tag, receives the prompt instruction, the buzzer on the positioning device, i.e., the tag, sounds, so that the mechanic knows that he has arrived at the item to be detected, preventing the mechanic from being too close or too far from the item to be detected, reducing the risk of accidentally touching other equipment or devices near the item to be detected, and improving the standardization of image acquisition.

[0038] After the electronic device determines that the mechanic has arrived at the item to be inspected, the viewing angle of the real-time video changes with the mechanic's movements. Therefore, by analyzing the real-time video, it is determined whether the mechanic has reached the preset collection posture for collecting the item to be inspected. The preset collection posture is the posture that the mechanic needs to maintain when collecting the image of the item to be inspected.

[0039] For example, if an item to be inspected is located on the top of the train, such as the collection of the car number, the mechanic needs to keep his head up; if an item to be inspected is a device at the bottom of the train, the mechanic needs to squat to collect the image, so the mechanic's posture can be determined based on the real-time video.

[0040] S103: If the preset acquisition posture is met, image information of the item to be detected is obtained.

[0041] The image information is collected by the target object through an image collection device.

[0042] For the embodiment of the present application, if the electronic device detects that the mechanic has reached a preset collection posture, it means that the mechanic's current posture meets the requirements of the subsequent image collection. The mechanic carries an image collection device, such as a camera, a mobile phone, etc. The image collection device is connected to the electronic device wirelessly. The mechanic can collect image information of the item to be inspected through the image collection device, and then the electronic device obtains the collected image information.

[0043] S104, determining the data collection quality of the target object for the item to be detected based on the image information.

[0044] For the embodiment of the present application, after the electronic device acquires the image information, if the image information is not clear enough, there are jittery afterimages, missing elements, etc., it may result in the inability to know the specific situation of the item to be inspected based on the image information, and the collected image information is invalid. Therefore, the electronic device determines the quality of the mechanic's collection of the item to be inspected based on the image information.

[0045] S105: If the data acquisition quality is lower than a preset quality threshold, the image acquisition device is controlled to re-acquire image information of the item to be detected.

[0046] For the embodiment of the present application, the preset quality threshold is used as a dividing point for the collected image information with a higher or lower quality. After the electronic device determines the data collection quality, it compares the data collection quality with the preset quality threshold. If it is lower than the preset quality threshold, it means that the image information is poor and the specific situation of the project cannot be determined based on the image information. The electronic device can send a collection instruction to the image collection device so that the image collection instruction re-collects the image information of the project. By re-collecting the image information of the project when the image information quality is low, the efficiency is higher than re-collecting after discovering the low quality, and a prompt message is output when the mechanic reaches the position of the item to be inspected, reducing the risk of omission.

[0047] In a possible implementation of the embodiment of the present application, in step S102, the real-time video is analyzed to determine whether the target object meets the preset collection posture of the item to be detected, which specifically includes step S1021 (not shown in the figure), step S1022 (not shown in the figure), step S1023 (not shown in the figure) and step S1024 (not shown in the figure), wherein: S1021: Perform feature recognition on the real-time video to determine whether the first identifier is recognized.

[0048] For the embodiments of the present application, the first identification can be a device, equipment, etc. near the item to be detected, or it can be a sticker, pendant, or other identification set in advance by the staff near the item to be detected. The electronic device inputs the real-time video into the trained network model for feature recognition, thereby determining whether the first identification is recognized. The network model can be a convolutional neural network model, a recurrent neural network model, or other types of network models. First, the staff can prepare a training sample set for the first identification, which includes images of the first identification at multiple angles, multiple heights, and under different lighting conditions of the first identification, and corresponds to the label of the first identification, and then inputs it into the initial network model for training to obtain a trained network model.

[0049] S1022: If the first marker is recognized, determine the position and outline of the first marker in the picture of the real-time video.

[0050] For the embodiment of the present application, after the electronic device recognizes the first identifier, it is necessary to determine the mechanic's posture according to the specific display of the first identifier in the real-time video. Therefore, the electronic device determines the position and outline of the first identifier in the real-time video. The electronic device can use the center position of the first identifier to represent the overall position of the first identifier. The electronic device draws a coordinate system with the lower left corner of the real-time video screen as the origin, thereby determining the coordinates representing the position of the first identifier. The electronic device performs edge detection on the real-time video, first denoises the real-time video and performs grayscale transformation, and determines the place where the grayscale value step occurs between the first identifier and the surrounding pixels in the grayscale image after the grayscale transformation as the outline of the first identifier.

[0051] S1023: If the first mark is located in a preset area in the picture, determine the area of ​​the outline.

[0052] For the embodiment of the present application, the preset area can be a circular area, a square area, or an area of ​​other shapes. The preset area is mapped in the coordinate system mentioned in step S1022, and then the first mark is determined to be within the preset area based on whether the coordinates of the center position of the first mark are within the preset area. After the electronic device determines the outline of the first mark, it counts the pixels within the outline range and uses the number of pixels to represent the area of ​​the outline.

[0053] If the first mark is located in the preset area, it means that the mechanic's perspective and orientation meet the standard posture requirements. If the first mark is not located in the preset area, it means that the mechanic's perspective and orientation do not meet the posture requirements. If the first mark is located in the preset area, it is necessary to determine the mechanic's posture height and the distance from the first mark. The area of ​​the first mark outline represents the mechanic's posture height and the distance from the mechanic to the first mark. The mechanic's posture height and the distance from the first mark also represent whether it meets the standard posture requirements.

[0054] S1024, determining a first ratio of the area to a preset area threshold, and if the first ratio is within a preset ratio interval, determining that the target object reaches a preset collection posture of the item to be detected.

[0055] For the embodiment of the present application, the preset area threshold is used as the area size of the target object's posture height and the distance from the first mark that meet the standard. The electronic device determines the ratio of the contour area to the preset area threshold. The closer the ratio is to 1, that is, the more consistent the contour area is with the preset area threshold, the more appropriate and standard the posture height and the distance from the first mark are. The preset ratio interval is used as the allowable error range. If the first ratio is within the preset ratio interval, it means that the posture height and the distance from the first mark meet the standard, and the mechanic's perspective and orientation also meet the standard, so the electronic device can determine that the target object has reached the preset acquisition posture. It is more accurate to determine whether the mechanic meets the preset acquisition posture by determining the contour area of ​​the first mark in the real-time video and whether it is located in the preset area. Meeting the preset acquisition posture makes it easier for the mechanic to capture images and is not easy to accidentally touch other equipment or devices around the item to be detected.

[0056] In a possible implementation of the embodiment of the present application, in step S104, the data collection quality of the target object for the item to be detected is determined based on the image information, specifically including step S1041 (not shown in the figure), step S1042 (not shown in the figure), step S1043 (not shown in the figure), step S1044 (not shown in the figure), step S1045 (not shown in the figure) and step S1046 (not shown in the figure), wherein: S1041, performing feature recognition on the image information to determine whether a second identifier exists in the image information.

[0057] The second identification is set in a preset area of ​​the item to be detected, and there are multiple second identifications, each of which has a different shape.

[0058] For the embodiment of the present application, the second identifier can be a sticker or a pendant, etc. The staff will pre-set the second identifier at a position near the item to be collected. After the mechanic meets the preset collection posture, he holds an image collection device, such as a camera, a mobile phone, etc. to collect image information. The second identifier is used to standardize the framing range of the image information and the quality of the image information. The electronic device can input the image information into a trained network model for feature recognition, so as to determine whether the second identifier exists in the image information. The network model can be a convolutional neural network model, a recurrent neural network model, or other types of network models. For specific means of training the network model, please refer to the content in step S1021.

[0059] S1042: If there are any second markers, determine a first number of second markers in the image information, and perform edge detection on the image information to obtain a contour of each second marker and a sub-image of each second marker.

[0060] For the embodiment of the present application, when the electronic device determines that there is a second identifier, the number of second identifiers in the image information is counted to obtain the first number. The more second identifiers there are in the image, the larger and more accurate the range of items to be detected covered by the image information is, and the less likely it is to cause data loss in the image information. The electronic device then performs edge detection on the image information to obtain the outline of each second identifier in the image information. After determining the outline, the image information can be segmented according to the outline to obtain a sub-image of the second identifier. For details on edge detection, please refer to the contents recorded in step S1022.

[0061] S1043, determining the similarity between the outline of each second mark and the corresponding standard outline, and determining the definition difference between the sub-image of each second mark and the corresponding standard image.

[0062] For the embodiment of the present application, after the electronic device determines the outline of the second identifier in the image information, it calculates the similarity between the outline and the preset outline of the corresponding second identifier, where the preset outline is the outline of the display shape of the second identifier when the quality meets the standard. Specifically, the electronic device can determine the similarity by calculating the cosine distance between the outline and the preset outline, or by using a pre-trained CNN model through a convolutional neural network to extract high-level features of the outline and the preset outline, and then calculate the distance between the feature vectors, such as cosine similarity, Euclidean distance, etc., to measure the similarity of the images. The higher the similarity, the closer the outline of the second identifier in the image information taken by the mechanic is to the preset outline, and there is no severe shaking or shaking during shooting that causes image distortion and blur, and the image information quality is higher. The electronic device determines the similarity corresponding to each second identifier in the image information in the above manner. The clarity of the standard image can be characterized by features such as contrast, edge sharpness, and signal-to-noise ratio, or by weighted calculation of contrast, edge sharpness, and signal-to-noise ratio to obtain a feature value, and the feature value is used to characterize the clarity. The electronic device analyzes the sub-image of each second identifier to obtain features such as contrast, edge sharpness, and signal-to-noise ratio of the sub-image, and similarly obtains a feature value by weighted calculation of contrast, edge sharpness, and signal-to-noise ratio, and the feature value is used to characterize the clarity. Then the electronic device subtracts the clarity of the second identifier in the image information from the clarity of the standard image corresponding to each second identifier to obtain a clarity difference. Among them, contrast: images with high contrast often look clearer, and the contrast of the image can be calculated, such as the difference between the maximum brightness and the minimum brightness. Edge sharpness: The clarity of the image is measured by detecting the edges in the image and calculating the sharpness of the edges. For example, an edge detection algorithm (such as Canny edge detection) can be used to detect the edges, and then the gradient amplitude of the edges can be calculated. Signal-to-noise ratio (SNR): images with high signal-to-noise ratio are usually clearer. Calculate the ratio of image signal power to noise power. The smaller the clarity difference is, the closer the clarity is, the clearer the second logo in the image is, and the higher the quality is.

[0063] S1044: Determine whether each second mark is located in a corresponding preset area in the picture.

[0064] For the embodiment of the present application, the preset area of ​​each second identification is set according to the screen of the image acquisition device, and the positional relationship between the preset areas is consistent with the positional relationship of the second identification at the item to be detected. When the mechanic holds the image acquisition device to take pictures, some second identifications may not be in the corresponding preset area due to the shooting angle, improper holding of the image acquisition device, etc., resulting in low shooting quality. Therefore, the electronic device determines that each second identification is in the corresponding preset area. The method of determination can refer to the content disclosed in step S1023.

[0065] S1045: If there is a target second identifier located in the corresponding preset area, determine the second number of the target second identifiers.

[0066] For the embodiment of the present application, the electronic device determines that there is a target second identification located in the corresponding preset area, and then counts the number of target second identifications to obtain a second number. The larger the second number, the more standardized the image shooting angle and position are, and the higher the image acquisition quality is.

[0067] S1046, determining the data acquisition quality based on the first quantity, the second quantity, the similarity, and the clarity difference.

[0068] For the embodiments of the present application, in summary, the first quantity, the second quantity, the similarity and the clarity are all key factors affecting the quality of image information acquisition. Therefore, it is more accurate for the electronic device to comprehensively determine the data acquisition quality based on the first quantity, the second quantity, the similarity and the clarity.

[0069] In a possible implementation of the embodiment of the present application, the data acquisition quality is determined based on the first quantity, the second quantity, the similarity, and the clarity difference in step S1046, specifically including step S1 (not shown in the figure), step S2 (not shown in the figure), step S3 (not shown in the figure), and step S4 (not shown in the figure), wherein: S1, determining a second ratio of the first quantity to the quantity of second identifiers corresponding to the items to be detected.

[0070] For the embodiment of the present application, the number of second identifiers set for each item to be detected may be different, and the electronic device calculates the ratio of the first number to the number of second identifiers of the current item to be detected to obtain a second proportion. The larger the second proportion, the more second identifiers are included in the image information, the more complete the items to be detected are, the less likely data is to be missing, and the higher the collection quality.

[0071] S2, determining a third ratio of the second quantity to the first quantity.

[0072] For the embodiment of the present application, a greater number of target second identifications appearing in the image information indicates that the position of the item to be detected in the image information is more standard and more regulated, which facilitates the subsequent determination of whether the item to be detected is abnormal from the image information. Therefore, a greater number of second identifications indicates a higher acquisition quality. Therefore, the electronic device determines the ratio of the second number to the first number, that is, the third ratio. The greater the third ratio, the higher the acquisition quality.

[0073] S3, determining a first average value of the similarities and a second average value of the clarity differences.

[0074] For the embodiment of the present application, after the electronic device determines the similarity of each second identifier in the image information, it determines the first average value through the average value calculation formula, and the first average value is used to more accurately represent the overall similarity. Similarly, after the electronic device determines the clarity difference of each second identifier in the image information, it determines the second average value through the average value calculation formula, and the second average value is used to more accurately represent the overall clarity difference.

[0075] S4, determining a first score based on the second proportion, the third proportion, the first average value, and the second average value.

[0076] The first score represents the data collection quality of the target object for the item to be detected.

[0077] For the embodiments of the present application, the second proportion, the third proportion, the first average value and the second average value are all important information that affect the quality of data collection, and the degree of influence is different. Therefore, the staff can set the coefficients corresponding to the second proportion, the third proportion, the first average value and the second average value in advance, and then the electronic device calls the corresponding coefficients for weighted calculation to obtain the first score, which characterizes the quality of data collection. It is more accurate to comprehensively determine the collection quality through the second proportion, the third proportion, the first average value of similarity and the second average value of clarity difference.

[0078] In a possible implementation of the embodiment of the present application, in step S105, the image acquisition device is controlled to re-acquire image information of the item to be detected, which specifically includes step S1051 (not shown in the figure) and step S1052 (not shown in the figure), wherein: S1051, controlling the image acquisition device to display the preset area of ​​each second mark, and detecting in real time whether each second mark is located in the corresponding preset area.

[0079] For the embodiment of the present application, a display screen is provided on the image acquisition device for displaying images, and the electronic device sends a control instruction to the image acquisition device. After receiving the control instruction, the image acquisition device displays the preset area of ​​each second mark on the display screen. The mechanic uses the displayed preset area to facilitate moving the image acquisition device to a suitable position so that each second mark is located in the corresponding preset area. During the movement of the image acquisition device, the electronic device detects in real time whether the second mark is located in the corresponding preset area. Specifically, the center position coordinates of each second mark can be determined, and the position of each second mark is represented by the center position coordinates. Whether the second mark is located in the preset area is determined by judging whether the center position coordinates are in the preset area.

[0080] In other embodiments, the electronic device may also generate an indication strategy based on the position relationship between the positions of all second identifiers in the image information and the corresponding preset areas in the picture, and send the indication strategy to the image acquisition device. The image acquisition device displays an arrow on the display screen according to the indication strategy, so that the mechanic moves the image acquisition device according to the arrow and reaches the appropriate position.

[0081] S1052: When it is detected that each second identification is located in the corresponding preset area and the time period for which each second identification is located in the corresponding preset area reaches a preset time period, the image acquisition device is controlled to acquire image information of a preset number of items to be detected.

[0082] The preset number is determined based on the similarity and clarity difference.

[0083] For the embodiment of the present application, when the electronic device detects that each second mark is located in the corresponding preset area, it means that the current position standard of the image acquisition device can collect qualified standard image information of the item to be detected, and at the same time, timing is performed to determine whether the time period in which each mark is located in the corresponding preset area reaches the preset time period. When the preset time period is reached, it means that the position of the image acquisition device tends to be stable and can collect high-quality image information of the item to be detected. After the electronic device determines the preset number of image information to be recollected based on the similarity and clarity difference, the preset number can be included in the control instruction in step S1051 and sent to the image acquisition device, so that the image acquisition device starts to collect the preset number of images when the time period in which the second marks are located in the corresponding preset area reaches the preset time period.

[0084] A possible implementation of the embodiment of the present application is to determine the preset number, specifically including step Sa (not shown in the figure), step Sb (not shown in the figure), step Sc (not shown in the figure), step Sd (not shown in the figure) and step Se (not shown in the figure), wherein: Sa, determine a third number of the first target second identifiers and an average similarity value corresponding to the first target second identifiers.

[0085] The first target second identifier includes a second identifier whose similarity is lower than a preset similarity threshold.

[0086] For the embodiment of the present application, if the similarity is lower than the preset similarity threshold, it means that the first target and the second identifier are blurred and deformed to a large extent, and the difficulty in identifying useful data from the image information is higher, and the more it is necessary to reshoot and shoot multiple image information. The more reshoots are done, the clearer the image can be determined from the reshot images, so that useful data can be more easily identified later. Therefore, the electronic device calculates the average similarity value of the first target and the second identifier through the average value calculation formula. The lower the similarity average value, the worse the quality of the current image information. And the first target and the second identifier are counted to obtain a third number. The larger the third number, the worse the quality of the current image information, and the more it is necessary to reshoot multiple image information.

[0087] Sb, determining a fourth number of second target second identifiers and an average value of clarity differences corresponding to the second target second identifiers.

[0088] The second target second identifier includes a second identifier whose clarity difference reaches a preset clarity difference threshold.

[0089] For the embodiment of the present application, if the clarity difference is lower than the preset clarity difference threshold, it means that the blurrier the second target and the second identifier are, the more difficult it is to identify useful data, and the more it is necessary to reshoot and take multiple images. The more reshoots are taken, the clearer the image can be determined from the reshot images, so that useful data can be more easily identified later. Therefore, the electronic device calculates the average clarity difference value of the second target and the second identifier through the average value calculation formula. The higher the average clarity difference value, the worse the quality of the current image information. And the second identifier of the second target is counted to obtain a fourth number. The larger the fourth number, the more areas with poor image quality in the current image information, the worse the image information quality, and the more it is necessary to reshoot multiple images.

[0090] Sc, determine the fourth ratio of the similarity average value to the third number and the fifth ratio of the clarity difference average value to the fourth number.

[0091] For the embodiment of the present application, the electronic device calculates the fourth ratio of the average similarity value to the third number, with the third number as the denominator and the average similarity value as the numerator. The smaller the fourth ratio is, the worse the image information quality is, and the more images need to be retaken. The electronic device takes the reciprocal of the average clarity difference value. The electronic device calculates the fifth ratio of the average clarity difference value to the fourth number, with the fourth number as the denominator and the reciprocal of the average clarity difference value as the numerator. The smaller the fifth ratio is, the worse the image information quality is, and the more images need to be retaken.

[0092] Sd, determine the second score based on the fourth proportion, the fifth proportion and their respective corresponding coefficients.

[0093] For the embodiment of the present application, in summary, the fourth proportion and the fifth proportion are both key factors affecting the number of retaken images, and the degree of influence is different, so the staff can set the coefficients corresponding to the fourth proportion and the fifth proportion in advance. After the electronic device determines the fourth proportion and the fifth proportion, it calls the corresponding coefficients to perform weighted calculation to obtain the second score.

[0094] Se, a preset number is determined based on the second score.

[0095] For the embodiment of the present application, the smaller the second score is, the more reshoots are needed. A plurality of preset score intervals may be pre-stored in the electronic device, and each preset score interval corresponds to a suitable number of reshoots. After the electronic device determines the preset score interval where the second score is located, the number of reshoots corresponding to the preset score interval is determined as the preset number. It is more accurate to comprehensively determine the preset number of reshoots by using the second identifier of the first target with a low similarity, the second identifier of the second target with an average clarity difference reaching a preset difference, the fourth proportion, and the fifth proportion.

[0096] In a possible implementation of the embodiment of the present application, step S105 further includes step S106 (not shown in the figure), step S107 (not shown in the figure), step S108 (not shown in the figure), and step S109 (not shown in the figure), wherein: S106, preprocessing the image information to obtain preprocessed image information.

[0097] According to the embodiment of the present application, the electronic device pre-processes the acquired image information, such as resizing, noise filtering, and color space conversion, so as to facilitate subsequent element recognition and text recognition.

[0098] S107, performing element detection on the preprocessed image information to obtain target elements.

[0099] For the embodiments of the present application, the electronic device may use a target detection algorithm to detect target elements in the image information, where the target elements are elements corresponding to the current detection items, such as various instruments and display screens. The electronic device uses a deep learning model to locate and mark objects in the image. The target detection algorithm can identify the main objects in the image and their bounding boxes, and output the detected target element positions. For example, the status of the control panel facilities in the train cab, the detection of the indication of the pantograph being raised and not raised, etc.

[0100] S108, performing text recognition on the pre-processed image information to obtain text information.

[0101] For the embodiment of the present application, the electronic device can use OCR (Optical Character Recognition) technology and algorithms to recognize the text part in the image information. The electronic device first crops the target detected area, and then inputs the cropped area into the text recognition algorithm model to extract the text information in the image information. For example, the fault page information of the monitoring screen in the vehicle monitoring room, the collection of the vehicle internal car number, etc.

[0102] S109: judging whether the to-be-detected item corresponding to the image information is abnormal based on the text information and the target element.

[0103] For the embodiment of the present application, the electronic device classifies the target elements in the image information, such as product labels, text descriptions, graphic elements, etc., according to the detection results of the elements and text in step S107 and step S108. These elements are further processed using custom business logic, such as extracting useful text information or processing specific graphic data, and the determined text information and target elements are compared with the text information and target elements in a normal state, so as to determine whether the detection items corresponding to the image information are abnormal. Correct results are returned for normal image information that meets the requirements, and wrong results are returned for abnormal images that are not required.

[0104] The above embodiment introduces an image-based method for detecting railway operation behavior from the perspective of method flow, and the following embodiment introduces an image-based device for detecting railway operation behavior from the perspective of a virtual module or a virtual unit. For details, please refer to the following embodiment.

[0105] The embodiment of the present application provides a device 20 for detecting railway operation behavior based on an image, such as Figure 2 As shown, the device 20 for detecting railway operation behavior based on images may specifically include: The data acquisition module 201 is used to acquire the real-time video of the target object in the preset area during the activity, the position of the target object in the train, and the nearest item to be detected. The real-time video is acquired by the video acquisition device on the target object; The judgment module 202 is used to output a prompt message when the position reaches the position corresponding to the item to be detected, and analyze the real-time video to determine whether the target object meets the preset collection posture of the item to be detected; The image acquisition module 203 is used to acquire image information of the item to be detected if the preset acquisition posture is met, and the image information is acquired by the target object through the image acquisition device; A quality determination module 204, used to determine the data collection quality of the target object for the item to be detected based on the image information; The control module 205 is used to control the image acquisition device to re-acquire image information of the item to be detected when the data acquisition quality is lower than a preset quality threshold.

[0106] The embodiment of the present application discloses a device 20 for detecting railway operation behavior based on images, wherein a data acquisition module 201 acquires real-time video, position and the nearest item to be detected during the activity of a target object. When it is determined that the target object has arrived at the position of the item to be detected, a judgment module 202 outputs a prompt message to remind the target object to arrive at the item to be detected, so that the target object is not likely to miss or miss the item to be detected. Since the real-time video is collected by a video acquisition device on the target object, the viewing angle of the real-time video changes with the movement of the target object. According to the real-time video of the judgment module 202, it can be judged whether the target object meets the requirements. The corresponding posture when collecting image information of the item to be detected is the preset collection posture. If it meets the preset collection posture, it means that the next image collection operation can be carried out. The image acquisition module 203 obtains the image information of the target object collected by the image acquisition device. The quality determination module 204 determines the data collection quality of the target object according to the image information. If the data acquisition device is lower than the preset quality threshold, it means that the display effect of the item to be detected in the image information is poor, and the specific situation of the item to be detected cannot be known. Therefore, the control module 205 re-controls the image acquisition device to collect the image information of the item to be detected, which is more efficient than re-shooting when the photo is blurred.

[0107] In a possible implementation of the embodiment of the present application, when the judgment module 202 analyzes the real-time video to determine whether the target object meets the preset collection posture of the item to be detected, it is specifically used to: Performing feature recognition on the real-time video to determine whether the first identifier is recognized; If the first marker is identified, determining the position and outline of the first marker in the picture of the real-time video; If the first mark is located in a preset area in the picture, determining the area of ​​the outline; A first ratio of the area to a preset area threshold is determined, and if the first ratio is within a preset ratio interval, it is determined that the target object reaches a preset collection posture of the item to be detected.

[0108] In a possible implementation of the embodiment of the present application, when the quality determination module 204 determines the data collection quality of the target object for the item to be detected based on the image information, it is specifically used to: Performing feature recognition on the image information to determine whether there is a second mark in the image information, the second mark is set in a preset area of ​​the item to be detected, and there are multiple second marks, each of which has a different shape; If the second marker exists, determining a first number of the second markers in the image information, and performing edge detection on the image information to obtain a contour of each second marker and a sub-image of each second marker; Determine the similarity between the outline of each second mark and the corresponding standard outline, and determine the definition difference between the sub-image of each second mark and the corresponding standard image; Determining whether each second mark is located in a corresponding preset area in the picture; If there is a target second identifier located in the corresponding preset area, determining a second number of target second identifiers; The data acquisition quality is determined based on the first quantity, the second quantity, the similarity, and the clarity difference.

[0109] In a possible implementation of the embodiment of the present application, when the quality determination module 204 determines the data acquisition quality based on the first quantity, the second quantity, the similarity, and the clarity difference, it is specifically used to: Determine a second ratio of the first quantity to the quantity of second identifiers corresponding to the items to be detected; Determine a third ratio of the second quantity to the first quantity; Determining a first average of the similarities and a second average of the clarity differences; A first score is determined based on the second proportion, the third proportion, the first average value, and the second average value, where the first score represents the data collection quality of the target object for the item to be detected.

[0110] In a possible implementation of the embodiment of the present application, the control module 205 is specifically used to control the image acquisition device to re-acquire the image information of the item to be detected: Controlling the image acquisition device to display the preset area of ​​each second mark, and detecting in real time whether each second mark is located in the corresponding preset area; When it is detected that each second identification is located in the corresponding preset area and the time period for which they are located in the corresponding preset area reaches the preset time period, the image acquisition device is controlled to acquire a preset number of image information of the items to be detected, and the preset number is determined based on the similarity and clarity difference.

[0111] In a possible implementation manner of the embodiment of the present application, the device 20 determines the preset number and further includes: A first determination module, configured to determine a third number of first target second identifiers and an average similarity value corresponding to the first target second identifiers, wherein the first target second identifiers include second identifiers whose similarity is lower than a preset similarity threshold; A second determination module is used to determine a fourth number of second target second identifiers and an average value of clarity differences corresponding to the second target second identifiers, wherein the second target second identifiers include second identifiers whose clarity differences are lower than a preset clarity difference threshold; A third determination module is used to determine a fourth ratio of the average similarity value to the third number and a fifth ratio of the average clarity difference value to the fourth number; A fourth determination module, configured to determine a second score based on the fourth proportion, the fifth proportion, and respective corresponding coefficients; The fifth determining module is used to determine a preset number based on the second score.

[0112] In a possible implementation of the embodiment of the present application, the device 20 further includes: A preprocessing module, used for preprocessing image information to obtain preprocessed image information; An element detection module is used to perform element detection on the preprocessed image information to obtain the target element; A text recognition module is used to perform text recognition on the pre-processed image information to obtain text information; The abnormality judgment module is used to judge whether the item to be detected corresponding to the image information is abnormal based on the text information and the target element.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described image-based device 20 for detecting railway operation behavior can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0114] An electronic device is provided in an embodiment of the present application, such as Figure 3 As shown, Figure 3 The electronic device 30 shown includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 30 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 30 does not constitute a limitation on the embodiments of the present application.

[0115] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0116] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but it does not mean that there is only one bus or only one type of bus.

[0117] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0118] The memory 303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the contents shown in the above method embodiment.

[0119] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0120] The present application embodiment provides a computer-readable storage medium, which stores a computer program, and when it runs on a computer, the computer can execute the corresponding content in the aforementioned method embodiment. Compared with the related art, in the present application embodiment, the real-time video, position and the nearest item to be detected during the activity of the target object are obtained, and after determining the position of the target object arriving at the item to be detected, a prompt message is output to remind the target object to arrive at the item to be detected, so that the target object is not easy to miss and miss the item to be detected. Since the real-time video is collected by the video acquisition device on the target object, the viewing angle of the real-time video changes with the action of the target object. According to the real-time video, it can be judged whether the target object meets the corresponding posture when the image information of the item to be detected is collected, that is, the preset collection posture. If it meets the preset collection posture, it means that the next step of image acquisition operation can be carried out, and the image information collected by the target object through the image acquisition device is obtained, and the data acquisition quality of the target object is determined according to the image information. If the data acquisition device is lower than the preset quality threshold, it means that the display effect of the item to be detected in the image information is poor, and the specific situation of the item to be detected cannot be known, so the image acquisition device is controlled again to collect the image information of the item to be detected, which is more efficient than reshooting after the photo is blurred.

[0121] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0122] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting railway operation behavior based on an image, characterized in that: include: Acquire a real-time video of a target object in a preset area during its activity, the position of the target object in the train, and an item to be detected closest to the position, wherein the real-time video is acquired by a video acquisition device on the target object; When the position reaches the position corresponding to the item to be detected, a prompt message is output, and the real-time video is analyzed to determine whether the target object meets the preset collection posture of the item to be detected; If the preset acquisition posture is met, image information of the item to be detected is acquired, where the image information is acquired by the target object through an image acquisition device; Determining the data acquisition quality of the target object for the item to be detected based on the image information; If the data acquisition quality is lower than a preset quality threshold, the image acquisition device is controlled to re-acquire image information of the item to be detected.

2. The method for detecting railway operation behavior based on images according to claim 1, characterized in that: The step of analyzing the real-time video to determine whether the target object meets the preset collection posture of the item to be detected includes: Performing feature recognition on the real-time video to determine whether a first identifier is recognized; If the first marker is identified, determining the position and outline of the first marker in the picture of the real-time video; If the first mark is located in a preset area in the picture, determining the area of ​​the outline; A first ratio of the area to a preset area threshold is determined, and if the first ratio is within a preset ratio interval, it is determined that the target object has reached a preset collection posture of the item to be detected.

3. The method for detecting railway operation behavior based on images according to claim 2, characterized in that: The determining, based on the image information, the data acquisition quality of the target object for the item to be detected includes: Performing feature recognition on the image information to determine whether a second mark exists in the image information, the second mark being set in a preset area of ​​the item to be detected, and the number of the second marks being multiple, each of which has a different shape; If the second marker exists, determining a first number of the second markers in the image information, and performing edge detection on the image information to obtain a contour of each second marker and a sub-image of each second marker; Determine the similarity between the outline of each second mark and the corresponding standard outline, and determine the definition difference between the sub-image of each second mark and the corresponding standard image; Determining whether each second mark is located in a corresponding preset area in the picture; If there is a target second identifier located in the corresponding preset area, determining a second number of target second identifiers; The data acquisition quality is determined based on the first number, the second number, the similarity, and the clarity difference.

4. The method for detecting railway operation behavior based on images according to claim 3, characterized in that: The determining the data acquisition quality based on the first quantity, the second quantity, the similarity and the clarity difference includes: Determine a second ratio of the first quantity to the quantity of second identifiers corresponding to the items to be detected; Determining a third ratio of the second quantity to the first quantity; Determining a first average value of the similarities and a second average value of the clarity differences; A first score is determined based on the second proportion, the third proportion, the first average value, and the second average value, where the first score represents the data collection quality of the target object on the item to be detected.

5. The method for detecting railway operation behavior based on images according to claim 1, characterized in that: The controlling the image acquisition device to re-acquire the image information of the item to be detected comprises: Controlling the image acquisition device to display the preset area of ​​each second mark, and detecting in real time whether each second mark is located in the corresponding preset area; When it is detected that each second identification is located in the corresponding preset area and the time for which they are all located in the corresponding preset area reaches a preset time, the image acquisition device is controlled to acquire a preset number of image information of the items to be detected, and the preset number is determined based on the similarity and clarity difference.

6. The method for detecting railway operation behavior based on images according to claim 5, characterized in that: The determining of the preset quantity includes: Determine a third number of first target second identifiers and an average value of similarities corresponding to the first target second identifiers, wherein the first target second identifiers include second identifiers whose similarity is lower than a preset similarity threshold; Determine a fourth number of second target second identifiers and an average value of clarity differences corresponding to the second target second identifiers, wherein the second target second identifiers include second identifiers whose clarity differences are lower than a preset clarity difference threshold; Determine a fourth ratio of the average similarity value to the third number and a fifth ratio of the average clarity difference value to the fourth number; Determine a second score based on the fourth proportion, the fifth proportion and their respective corresponding coefficients; The preset number is determined based on the second score.

7. The method for detecting railway operation behavior based on images according to claim 1, characterized in that: If the data collection quality reaches a preset quality threshold, the following steps are further included: Preprocessing the image information to obtain preprocessed image information; Performing element detection on the preprocessed image information to obtain target elements; Performing text recognition on the preprocessed image information to obtain text information; Based on the text information and the target element, it is determined whether the item to be detected corresponding to the image information is abnormal.

8. An image-based device for detecting railway operation behavior, characterized in that: include: A data acquisition module is used to acquire real-time video of a target object in a preset area during its activity, the position of the target object in the train, and the nearest item to be detected, wherein the real-time video is acquired by a video acquisition device on the target object; A judgment module, configured to output a prompt message when the position reaches the position corresponding to the item to be detected, and analyze the real-time video to determine whether the target object meets the preset collection posture of the item to be detected; An image acquisition module, used for acquiring image information of the item to be detected if the preset acquisition posture is met, wherein the image information is acquired by the target object through an image acquisition device; A quality determination module, used to determine the data acquisition quality of the target object for the item to be detected based on the image information; The control module is used to control the image acquisition device to re-acquire the image information of the item to be detected when the data acquisition quality is lower than a preset quality threshold.

9. An electronic device, characterized in that: It includes: at least one processor; Memory; At least one application, wherein the at least one application is stored in the memory and configured to be executed by the at least one processor, and the at least one application is used to execute the method for detecting railway operation behavior based on an image according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the method for detecting railway operation behavior based on an image as described in any one of claims 1 to 7.

Images