A method, apparatus, and device for generating flight time series based on image recognition
By using image recognition-based methods to automatically identify and encode feature points and marker lines in flight timing diagrams, the problems of long conversion time and high error rate in converting flight timing diagrams into executable programs are solved, achieving efficient and accurate flight timing generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 北京天兵科技有限公司
- Filing Date
- 2023-02-09
- Publication Date
- 2026-05-26
Smart Images
Figure CN116385726B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of spacecraft control system design, specifically to a method, apparatus, and device for generating flight timing based on image recognition. Background Technology
[0002] Spacecraft timing generally refers to the set of time-based program commands issued during flight, from spacecraft ignition to spacecraft-rocket separation. Taking a manned spacecraft as an example, during normal flight, the spacecraft issues timing commands for escape tower separation, booster separation, first and second stage separation, fairing separation, second stage engine separation, and spacecraft-rocket separation. Rocket flight involves many timing sequences, and any minor error can affect the success or failure of the rocket launch. Flight timing sequences are often drawn using specialized plotting software, and then software programmers need to convert the required timing sequences into executable programs based on the launch control process model and download them into the spacecraft control system.
[0003] Currently, converting flight timing diagrams into executable programs typically requires programmers to first understand the launch control flow and then manually convert it into an executable program. This process is time-consuming, inefficient, and has a high error rate due to its manual conversion nature. Summary of the Invention
[0004] This application provides a method, apparatus, and device for generating flight time series based on image recognition, in order to solve the problems of long time consumption, low efficiency, and high error rate in the current process of converting flight time series plots into executable programs.
[0005] In a first aspect, this application provides a flight timing generation method based on image recognition, the method comprising:
[0006] Obtain the original timing diagram and perform grayscale conversion on the original timing diagram to obtain the timing diagram to be processed;
[0007] Each feature point of the timing diagram to be processed is detected sequentially, and the horizontal time axis and at least one identifier line representing the timing program instruction of the timing diagram to be processed are identified based on the pixel position and gray value of each feature point, wherein one end of the at least one identifier line is located on the horizontal time axis.
[0008] The timing diagram to be processed is divided into at least one sub-region based on the horizontal time axis, the at least one marker line, and the pixel position corresponding to the at least one marker line;
[0009] The flight timing execution code is generated by encoding the text information and pixel positions within the at least one sub-region.
[0010] Optionally, the step of sequentially detecting each feature point of the time sequence diagram to be processed, and identifying the horizontal time axis and at least one marker line representing the time program instructions of the time sequence diagram to be processed based on the pixel position and grayscale value of each feature point, includes:
[0011] The feature points of the time sequence image to be processed are detected sequentially using an image feature point detection algorithm.
[0012] If a number of consecutive feature points with gray values within a preset range are detected, and the pixel ordinates of the number of feature points are the same, then the number of feature points constitutes the horizontal time axis.
[0013] If a number of consecutive feature points with gray values within a preset range are detected, and the pixel x-coordinates of the number of feature points are the same, then the number of feature points is the marker line.
[0014] Optionally, dividing the time sequence map to be processed into at least one sub-region based on the horizontal time axis, the at least one marker line, and the pixel positions corresponding to the at least one marker line includes:
[0015] The at least one marker line is divided into an upper marker line and a lower marker line according to the relative position of the at least one marker line with respect to the horizontal time axis, wherein the upper marker line is located above the horizontal time axis and the lower marker line is located below the horizontal time axis;
[0016] The area above the upper marker line, the area below the lower marker line, and the area between two adjacent marker lines are divided into at least one sub-region. The area above the upper marker line and the area below the lower marker line are time program instruction sub-regions, and the area between two adjacent marker lines is an instruction time interval sub-region.
[0017] Optionally, the area above the upper marker line, the area below the lower marker line, and the area between two adjacent marker lines are divided into at least one sub-region, including:
[0018] The area above the upper marker line to the top of the timing diagram to be processed is divided into a time program instruction sub-region corresponding to the upper marker line. The pixel ordinate of the lower end of the time program instruction sub-region is consistent with the pixel ordinate of the end of the upper marker line away from the horizontal time axis, and the two endpoints of the lower end are respectively located at the center of the upper marker line and the two adjacent marker lines.
[0019] The area below the lower identifier line to the bottom of the timing diagram to be processed is divided into a time program instruction sub-region corresponding to the lower identifier line. The pixel ordinate of the upper end of the time program instruction sub-region is consistent with the pixel ordinate of the end of the lower identifier line away from the horizontal time axis, and the two endpoints of the upper end are respectively located at the center of the lower identifier line and the two adjacent identifier lines.
[0020] The area between two adjacent marker lines is divided into instruction time interval sub-regions corresponding to the two adjacent marker lines. If the two adjacent marker lines are on the same side of the horizontal time axis, the four endpoints of the two adjacent marker lines are respectively used as the four endpoints of the instruction time interval sub-region. If the two adjacent marker lines are on opposite sides of the horizontal time axis, the ends of the two adjacent marker lines away from the horizontal time axis are used as the two diagonal endpoints of the instruction time interval sub-region. The pixel horizontal coordinates of the left and right ends of the time program instruction sub-region are consistent with the pixel horizontal coordinates of the two adjacent marker lines.
[0021] Optionally, before encoding based on the text information within the at least one sub-region and the pixel positions of the at least one sub-region, the method further includes:
[0022] The text information within each of the at least one sub-region is identified using a text recognition algorithm.
[0023] Optionally, encoding is performed based on the text information within the at least one sub-region and the pixel positions of the at least one sub-region to generate flight timing execution code, including:
[0024] The pixel coordinates of the upper left corner endpoint of the at least one sub-region are taken as the pixel position of the at least one sub-region;
[0025] The at least one sub-region is sorted in ascending order of pixel x-coordinates, and the text information within the at least one sub-region is encoded sequentially to generate flight timing execution code.
[0026] Secondly, this application provides a flight timing generation apparatus based on image recognition, the apparatus comprising:
[0027] The acquisition module is used to acquire the original timing diagram and perform grayscale conversion on the original timing diagram to obtain the timing diagram to be processed.
[0028] The detection module is used to sequentially detect each feature point of the time sequence diagram to be processed, and identify the horizontal time axis and at least one identifier line representing the time program instruction of the time sequence diagram to be processed based on the pixel position and gray value of each feature point, wherein one end of the at least one identifier line is located on the horizontal time axis.
[0029] The segmentation module is used to divide the time sequence diagram to be processed into at least one sub-region based on the horizontal time axis, the at least one marker line, and the pixel position corresponding to the at least one marker line.
[0030] The encoding module is used to perform flight timing encoding based on the text information within the at least one sub-region and the pixel positions of the at least one sub-region.
[0031] Thirdly, this application provides a flight timing generation device based on image recognition, comprising: at least one processor and at least one memory;
[0032] The memory stores program code, which, when executed by the processor, causes the processor to perform the following processes:
[0033] Obtain the original timing diagram and perform grayscale conversion on the original timing diagram to obtain the timing diagram to be processed;
[0034] Each feature point of the timing diagram to be processed is detected sequentially, and the horizontal time axis and at least one identifier line representing the timing program instruction of the timing diagram to be processed are identified based on the pixel position and gray value of each feature point, wherein one end of the at least one identifier line is located on the horizontal time axis.
[0035] The timing diagram to be processed is divided into at least one sub-region based on the horizontal time axis, the at least one marker line, and the pixel position corresponding to the at least one marker line;
[0036] The flight timing execution code is generated by encoding the text information and pixel positions within the at least one sub-region.
[0037] Fourthly, this application also provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.
[0038] Fifthly, this application also provides a computer program product, including a computer program, the computer program including program instructions, which, when executed by an electronic device, cause the electronic device to execute any of the above-described image recognition-based flight timing generation methods.
[0039] Furthermore, the technical effects of any of the implementation methods in the second to fifth aspects can be found in the technical effects of different implementation methods in the first aspect, and will not be repeated here.
[0040] The flight timing generation method, apparatus, and device based on image recognition provided by this invention have the following beneficial effects:
[0041] This application provides a flight time sequence generation method based on image recognition. It processes the drawn flight time sequence diagram through image processing to generate executable flight time sequence code, avoiding the time-consuming, inefficient, and error-prone process of manually converting flight time sequence diagrams into executable programs. By using region segmentation and text recognition methods on the flight time sequence diagram, real-time conversion of the flight time sequence diagram can be achieved, significantly increasing the conversion efficiency and success rate. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0043] Figure 1 A schematic diagram of a flight timing generation method based on image recognition provided in an embodiment of this application;
[0044] Figure 2 A schematic diagram of a flight timing sequence to be processed is provided for an embodiment of this application;
[0045] Figure 3 This application provides a schematic diagram of flight timing after dividing the region;
[0046] Figure 4 A schematic diagram of a flight timing generation structure based on image recognition is provided in an embodiment of this application;
[0047] Figure 5 This is a schematic diagram of a flight timing generation structure based on image recognition, provided as an embodiment of this application. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0049] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.
[0050] The application scenarios described in the embodiments of this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems. In the description of this invention, unless otherwise stated, "multiple" means two or more.
[0051] Flight timing diagrams are typically created using specialized plotting software. Software programmers then convert the required timing diagrams into executable programs based on the engine control flow model, which are then downloaded into the aircraft control system. During flight, the requirements for flight timing are very high, often necessitating repeated verification by various systems to create an executable flight timing program. Currently, the process of converting flight timing diagrams into executable programs usually requires programmers to first understand the engine control flow and then manually convert it into an executable program. This process is time-consuming, inefficient, and has a high error rate due to its manual conversion nature.
[0052] Based on this, this application provides a flight timing generation method based on image recognition. It detects the marker lines representing time-bound program instructions in the flight timing diagram using image recognition, identifies the meaning of each marker line and the time interval between adjacent time-bound program instructions, and then generates flight timing execution code. This enables real-time conversion of flight timing diagrams, greatly increasing the conversion efficiency and success rate, and avoiding the significant manpower required for manual conversion.
[0053] like Figure 1 As shown, an image recognition-based flight timing generation method provided in this application includes:
[0054] Step S101: Obtain the original timing diagram and perform grayscale conversion on the original timing diagram to obtain the timing diagram to be processed;
[0055] Currently, flight timing diagrams are typically drawn by professionals using tools such as AutoCAD or Visio. The industry's drawing rules are generally consistent: a horizontal line serves as the time axis across the entire image, with at least one vertical line distributed above or below it. One end of each vertical line lies on the horizontal time axis, and each vertical line represents one or more time-based program commands. Text is used to represent these commands above each vertical line, and the area between two vertical lines contains text information indicating the time interval between the commands. It should be noted that if two adjacent vertical lines are on opposite sides of the horizontal time axis, the text information indicating the time interval between the commands can, but is not limited to, be located on one side of one of the vertical lines. In this embodiment, the original flight timing diagram is first obtained. Then, a coordinate system can be established, but is not limited to, using the upper left corner of the image as the origin. Alternatively, the lower left corner of the image can also be used as the origin. The specific implementation process is determined by those skilled in the art based on actual needs and is not limited here. Figure 2 The diagram shown is a flight timing sequence diagram to be processed according to an embodiment of this application. Figure 2 The outer border is the frame of the flight time-series image, and the horizontal line in the middle is the horizontal time axis. Figure 2 The example shows the vertical lines corresponding to four time program instructions: opening valve A, opening valve B, opening valve C and opening valve D, and closing valve A and closing valve B. The time intervals corresponding to the four time program instructions are 1s, 0.4s, and 10s, respectively.
[0056] Step S102: Sequentially detect each feature point of the timing diagram to be processed, and identify the horizontal time axis and at least one identifier line representing the time program instruction of the timing diagram to be processed based on the pixel position and gray value of each feature point, wherein one end of the at least one identifier line is located on the horizontal time axis.
[0057] In the process of processing the timing diagram to be processed, this embodiment of the application first identifies the horizontal time axis and at least one marker line representing the timing program instructions. The marker line is usually represented by a vertical line with one end located on the horizontal time axis. The process of detecting the horizontal time axis and the marker line is as follows:
[0058] The image feature point detection algorithm sequentially detects each feature point of the time sequence image to be processed. If a continuous number of feature points with gray values within a preset range are detected, and the pixel ordinates of the continuous number of feature points are the same, then the continuous number of feature points are the horizontal time axis. If a continuous number of feature points with gray values within a preset range are detected, and the pixel abscissas of the continuous number of feature points are the same, then the continuous number of feature points are the marker lines.
[0059] In computers, images are composed of many pixels. Converting an image to grayscale allows each pixel to be represented by a value between 0 and 255, such as zero for black and 255 for white. This embodiment first converts the original time series image to grayscale to obtain the time series image to be processed. Then, it determines the horizontal coordinate axis and marker lines by detecting image feature points. Image feature points can be simply understood as prominent points in the image, such as contour points, bright spots in darker areas, and dark spots in brighter areas. Algorithms such as SIFT (Scale-invariant feature transform) can be used, but are not limited to, as SIFT can find all key points on the time series image to be processed, such as the starting points of lines, the corner points of letters and Chinese characters, etc. After detecting all feature points in the time series image to be processed, the horizontal time axis and marker lines are determined based on the pixel position and grayscale value of each feature point. Figure 2 As shown, the horizontal time axis in the time sequence diagram to be processed has the following characteristics: the pixel ordinates of all feature points on the horizontal time axis are the same, the pixel abscissas are continuous, and the gray values of all feature points on the horizontal time axis are close to zero. Based on this characteristic, if a continuous number of feature points with gray values within a preset range are detected, and the pixel ordinates of the feature points are the same, then the number of feature points is the horizontal time axis. It should be noted that the horizontal lines in Chinese characters also have the characteristics of the pixel ordinates of the feature points being the same, the pixel abscissas being continuous, and the gray values of all feature points being close to zero. Therefore, the first preset number is limited here to distinguish the horizontal lines of Chinese characters from the horizontal time axis. The lines in the time sequence diagram to be processed after grayscale conversion may not be completely black, but rather gray with gray values close to zero. Therefore, in this embodiment, a preset range is set for the gray values. The preset range and the first preset number can be determined by those skilled in the art based on actual conditions, and are not limited here.
[0060] Similarly, the marker lines representing time program instructions also have the following characteristics in the time sequence diagram to be processed: the pixel abscissas of all feature points on the marker line are the same, the pixel ordinates are continuous, and the gray values of all feature points on the marker line are close to zero. Based on this characteristic, if at least a second preset number of feature points with gray values within a preset range are detected, and at least a second preset number of feature points have the same pixel abscissa, then at least a second preset number of feature points are marker lines. It should be noted that vertical lines in Chinese characters also have the characteristics of the pixel ordinates of feature points being the same, the pixel abscissas being continuous, and the gray values of all feature points being close to zero. In addition, the flight time sequence diagram also includes some vertical lines that mark time periods. Therefore, the second preset number is limited here to distinguish the horizontal lines of Chinese characters, the aforementioned interference lines, and the horizontal time axis. Similarly, the second preset number can be determined by those skilled in the art based on actual conditions and is not limited here. As an optional implementation, the length of the vertical line can also be determined by the pixel coordinates of the detected feature points. When the length of the vertical line is greater than max(Ln) / 2, the marker line is identified, where Ln is the length of each vertical line detected on the horizontal time axis.
[0061] Step S103: Divide the time sequence map to be processed into at least one sub-region based on the horizontal time axis, at least one marker line and the pixel position corresponding to at least one marker line;
[0062] like Figure 2 As shown, each marker line is marked with text information indicating a time-based instruction program above or below it. The area above each marker line is divided into sub-regions representing different time-based instruction programs. Text recognition of these sub-regions accurately reveals the meaning of each marker line. Between adjacent marker lines, text information indicating the time interval between two adjacent time-based instructions is displayed. The area between adjacent marker lines is divided into sub-regions representing different instruction time intervals. Text recognition of these sub-regions accurately reveals the time interval between adjacent marker lines. The specific method for dividing the timing diagram to be processed into at least one sub-region based on the horizontal time axis, the at least one marker line, and the pixel position corresponding to the at least one marker line includes:
[0063] At least one marker line is divided into an upper marker line and a lower marker line based on its relative position to the horizontal time axis, wherein the upper marker line is located above the horizontal time axis and the lower marker line is located below the horizontal time axis; the area above the upper marker line, the area below the lower marker line, and the area between two adjacent marker lines are divided into at least one sub-region, wherein the area above the upper marker line and the area below the lower marker line are time program instruction sub-regions, and the area between two adjacent marker lines is instruction time interval sub-regions.
[0064] like Figure 2 As shown, the marker lines are located on the upper and lower sides of the horizontal time axis. Therefore, the marker lines are divided into upper and lower marker lines. The method for determining each sub-region is as follows:
[0065] The region above the upper marker line to the top of the timing diagram to be processed is divided into a time program instruction sub-region corresponding to the upper marker line. The ordinate of the lower pixel of this sub-region is the same as the ordinate of the pixel at the end of the upper marker line furthest from the horizontal time axis, and its two endpoints are located at the center of the upper marker line and two adjacent marker lines, respectively. The region below the lower marker line to the bottom of the timing diagram to be processed is also divided into a time program instruction sub-region corresponding to the lower marker line. The ordinate of the upper pixel of this sub-region is the same as the ordinate of the pixel at the end of the lower marker line furthest from the horizontal time axis, and its two endpoints are located at the center of the lower marker line and two adjacent marker lines, respectively. The region between two adjacent marker lines is divided into instruction time interval sub-regions corresponding to the two adjacent marker lines. If two adjacent marker lines are on the same side of the horizontal time axis, their four endpoints are used as the four endpoints of the instruction time interval sub-region. If two adjacent marker lines are on opposite sides of the horizontal time axis, their ends furthest from the horizontal time axis are used as the two diagonal endpoints of the instruction time interval sub-region, and the abscissas of the pixels at the left and right ends of the time program instruction sub-region are the same as the abscissas of the pixels of the two adjacent marker lines, respectively.
[0066] It should be noted that, in order to perform flight timing encoding subsequently, this embodiment of the application also needs to record the pixel coordinates of the starting point of the horizontal time axis, the pixel coordinates of the starting point of the marker line, and the pixel coordinates of the endpoints of the sub-regions. A schematic diagram of the divided sub-regions is shown below. Figure 3 As shown, where, Figure 3 The four time-based program instructions and the instruction time intervals given in the document are... Figure 2 The same applies, so I won't elaborate further. Sub-regions ① and ③ are time-based program instruction sub-regions, sub-region ② is the instruction time interval sub-region, (x0, y0) are the pixel coordinates of the starting point of the horizontal coordinate axis, and (x1, y1) are... Figure 3 The pixel coordinates (x2, y2) corresponding to the vertical line furthest from the horizontal time axis given in the first timing program instruction are... Figure 3 The pixel coordinates (x3, y3) corresponding to the vertical line furthest from the horizontal time axis in the second timing program instruction given in the text are: Figure 3 The figure shows the pixel coordinates of the vertical line corresponding to the third timing instruction, located away from the horizontal time axis. X and Y are the horizontal and vertical axes of the coordinate system established by the flight timing diagram, respectively. It should be noted that the origin of the coordinate system can be determined by those skilled in the art; this figure only provides one example.
[0067] Step S104: Encode the text information and pixel positions of at least one sub-region to generate flight timing execution code.
[0068] After dividing the region into sub-regions, this embodiment of the application also requires recognizing the text information within each sub-region to perform flight timing encoding based on the text information and its corresponding pixel coordinates. Specifically, this can be achieved, but is not limited to, using the OCR text recognition function in OpenCV to recognize text information in the sub-regions, such as numbers, letters, and Chinese characters. Optionally, the recognized numbers and letters can be assigned different meanings. For example, if the time program command corresponding to sub-regions ① and ③ is "open valve," then sub-regions ① and ③ are defined as "open." If sub-region ② corresponds to a command time interval, then sub-region ② is defined as "delay."
[0069] After identifying the text information within each sub-region, the flight timing is encoded based on the text information and the order of the sub-regions. Specifically, the sub-regions first need to be sorted according to the order of the time program instructions in the timing diagram to be processed. In this embodiment, the sorting can be based on the sub-regions corresponding to each time program instruction. As an optional implementation, the pixel coordinates of the upper left corner of at least one sub-region can be used as the pixel position of at least one sub-region. Alternatively, the horizontal coordinates of the lower left corner of at least one sub-region can be used as the pixel position of at least one sub-region. Those skilled in the art can set it according to actual needs, and this is not limited here. Specifically, at least one sub-region is sorted in ascending order of its horizontal pixel coordinates, and the text information within each sub-region is encoded sequentially to generate flight timing execution code. For example, for... Figure 3 The sub-regions in the image, ordered by their pixel x-coordinates from smallest to largest, are sub-region ①, sub-region ②, and sub-region ③. During flight timing encoding, sub-region ① is encoded first, executing the time program command to open valve A. Then, after waiting for the time represented by sub-region ② (i.e., 1 second), the time program command for sub-region ③ is executed, which opens valve B. This process completes the encoding of the sub-regions. Figure 3 The flight timing code shown is used to generate flight timing execution code.
[0070] This application provides a flight timing generation method based on image recognition. By dividing the flight timing diagram into sub-regions corresponding to different time program instructions and instruction time intervals, image recognition can be used to identify the text information in the sub-regions, automatically encode the flight timing, and generate executable flight timing code. This avoids the manual conversion process, greatly saves conversion time, improves conversion efficiency, and significantly reduces the error rate during the conversion process.
[0071] The above describes a flight timing generation method based on image recognition in an embodiment of the present invention. The following describes the apparatus for executing the flight timing generation method based on image recognition.
[0072] Please see Figure 4 This invention provides an image recognition-based flight timing generation device, comprising:
[0073] The acquisition module 401 is used to acquire the original timing diagram and perform grayscale conversion on the original timing diagram to obtain the timing diagram to be processed.
[0074] The detection module 402 is used to sequentially detect each feature point of the time sequence diagram to be processed, and identify the horizontal time axis and at least one identifier line representing the time program instruction of the time sequence diagram to be processed based on the pixel position and gray value of each feature point, wherein one end of the at least one identifier line is located on the horizontal time axis.
[0075] The segmentation module 403 is used to divide the time sequence map to be processed into at least one sub-region based on the horizontal time axis, the at least one marker line and the pixel position corresponding to the at least one marker line;
[0076] The encoding module 404 is used to perform flight timing encoding based on the text information within the at least one sub-region and the pixel position of the at least one sub-region.
[0077] Optional, the detection module 402 is specifically used for:
[0078] The feature points of the time sequence image to be processed are detected sequentially using an image feature point detection algorithm.
[0079] If a number of consecutive feature points with gray values within a preset range are detected, and the pixel ordinates of the number of feature points are the same, then the number of feature points constitutes the horizontal time axis.
[0080] If a number of consecutive feature points with gray values within a preset range are detected, and the pixel x-coordinates of the number of feature points are the same, then the number of feature points is the marker line.
[0081] Optionally, module 403 is used specifically for:
[0082] The at least one marker line is divided into an upper marker line and a lower marker line according to the relative position of the at least one marker line with respect to the horizontal time axis, wherein the upper marker line is located above the horizontal time axis and the lower marker line is located below the horizontal time axis;
[0083] The area above the upper marker line, the area below the lower marker line, and the area between two adjacent marker lines are divided into at least one sub-region. The area above the upper marker line and the area below the lower marker line are time program instruction sub-regions, and the area between two adjacent marker lines is an instruction time interval sub-region.
[0084] Optionally, module 403 is used specifically for:
[0085] The area above the upper marker line to the top of the timing diagram to be processed is divided into a time program instruction sub-region corresponding to the upper marker line. The pixel ordinate of the lower end of the time program instruction sub-region is consistent with the pixel ordinate of the end of the upper marker line away from the horizontal time axis, and the two endpoints of the lower end are respectively located at the center of the upper marker line and the two adjacent marker lines.
[0086] The area below the lower identifier line to the bottom of the timing diagram to be processed is divided into a time program instruction sub-region corresponding to the lower identifier line. The pixel ordinate of the upper end of the time program instruction sub-region is consistent with the pixel ordinate of the end of the lower identifier line away from the horizontal time axis, and the two endpoints of the upper end are respectively located at the center of the lower identifier line and the two adjacent identifier lines.
[0087] The area between two adjacent marker lines is divided into instruction time interval sub-regions corresponding to the two adjacent marker lines. If the two adjacent marker lines are on the same side of the horizontal time axis, the four endpoints of the two adjacent marker lines are respectively used as the four endpoints of the instruction time interval sub-region. If the two adjacent marker lines are on opposite sides of the horizontal time axis, the ends of the two adjacent marker lines away from the horizontal time axis are used as the two diagonal endpoints of the instruction time interval sub-region. The pixel horizontal coordinates of the left and right ends of the time program instruction sub-region are consistent with the pixel horizontal coordinates of the two adjacent marker lines.
[0088] Optionally, the device further includes a recognition module 405, used to recognize text information in the at least one sub-region using a character recognition algorithm.
[0089] Optional, encoding module 404, specifically used for:
[0090] The pixel coordinates of the upper left corner endpoint of the at least one sub-region are taken as the pixel position of the at least one sub-region;
[0091] The at least one sub-region is sorted in ascending order of pixel x-coordinates, and the text information within the at least one sub-region is encoded sequentially to generate flight timing execution code.
[0092] The above describes a flight timing generation device based on image recognition in the embodiments of this application from the perspective of modular functional entities. The following describes a flight timing generation device based on image recognition in the embodiments of this application from the perspective of hardware processing.
[0093] Please see Figure 5 This application embodiment includes an image recognition-based flight timing generation device, at least one processor 501 and at least one memory 502, and a bus system 509;
[0094] The memory stores program code, which, when executed by the processor, causes the processor to perform the following processes:
[0095] Obtain the original timing diagram and perform grayscale conversion on the original timing diagram to obtain the timing diagram to be processed;
[0096] Each feature point of the timing diagram to be processed is detected sequentially, and the horizontal time axis and at least one identifier line representing the timing program instruction of the timing diagram to be processed are identified based on the pixel position and gray value of each feature point, wherein one end of the at least one identifier line is located on the horizontal time axis.
[0097] The timing diagram to be processed is divided into at least one sub-region based on the horizontal time axis, the at least one marker line, and the pixel position corresponding to the at least one marker line;
[0098] The flight timing execution code is generated by encoding the text information and pixel positions within the at least one sub-region.
[0099] Figure 5 This is a schematic diagram of a flight timing generation device based on image recognition, provided in an embodiment of this application. The device 500 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 501 (e.g., one or more processors) and a memory 502, and one or more storage media 503 (e.g., one or more mass storage devices) for storing application programs 504 or data 505. The memory 502 and storage media 503 can be temporary or persistent storage. The program stored in the storage media 503 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on an information processing device. Furthermore, the processor 501 may be configured to communicate with the storage media 503 and execute the series of instruction operations in the storage media 503 on the device 500.
[0100] Device 500 may also include one or more wired or wireless network interfaces 507, one or more input / output interfaces 508, and / or one or more operating systems 506, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.
[0101] This invention also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform a flight timing generation method based on image recognition provided in the above embodiments.
[0102] This application also provides a computer program product, including a computer program, which includes program instructions. When the program instructions are executed by an electronic device, the electronic device executes a flight timing generation method based on image recognition provided in the above embodiments.
[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0104] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.
[0105] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0106] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.
[0107] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.
[0108] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0109] The technical solutions provided in this application have been described in detail above. Specific examples have been used in this application to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A flight timing generation method based on image recognition, characterized in that, include: Obtain the original timing diagram and perform grayscale conversion on the original timing diagram to obtain the timing diagram to be processed; Each feature point of the timing diagram to be processed is detected sequentially, and the horizontal time axis and at least one identifier line representing the timing program instruction of the timing diagram to be processed are identified based on the pixel position and gray value of each feature point, wherein one end of the at least one identifier line is located on the horizontal time axis. The timing diagram to be processed is divided into at least one sub-region based on the horizontal time axis, the at least one marker line, and the pixel position corresponding to the at least one marker line; Encode the text information and pixel positions within the at least one sub-region to generate flight timing execution code; The step of encoding the text information and pixel positions within the at least one sub-region to generate flight timing execution code specifically includes: The pixel coordinates of the upper left corner of the at least one sub-region are used as the pixel position of the at least one sub-region; the at least one sub-region is sorted in ascending order of pixel horizontal coordinates, and the text information in the at least one sub-region is encoded in turn to generate flight timing execution code.
2. The method according to claim 1, characterized in that, The step of sequentially detecting each feature point of the time sequence diagram to be processed, and identifying the horizontal time axis and at least one marker line representing the time program instructions of the time sequence diagram to be processed based on the pixel position and grayscale value of each feature point, includes: The feature points of the time sequence image to be processed are detected sequentially using an image feature point detection algorithm. If a number of consecutive feature points with gray values within a preset range are detected, and the pixel ordinates of the number of feature points are the same, then the number of feature points constitutes the horizontal time axis. If a number of consecutive feature points with gray values within a preset range are detected, and the pixel x-coordinates of the number of feature points are the same, then the number of feature points is the marker line.
3. The method according to claim 1, characterized in that, The step of dividing the time sequence map to be processed into at least one sub-region based on the horizontal time axis, the at least one marker line, and the pixel positions corresponding to the at least one marker line includes: The at least one marker line is divided into an upper marker line and a lower marker line according to the relative position of the at least one marker line with respect to the horizontal time axis, wherein the upper marker line is located above the horizontal time axis and the lower marker line is located below the horizontal time axis; The area above the upper marker line, the area below the lower marker line, and the area between two adjacent marker lines are divided into at least one sub-region. The area above the upper marker line and the area below the lower marker line are time program instruction sub-regions, and the area between two adjacent marker lines is an instruction time interval sub-region.
4. The method according to claim 3, characterized in that, Dividing the area above the upper marker line, the area below the lower marker line, and the area between two adjacent marker lines into at least one sub-region includes: The area above the upper marker line to the top of the timing diagram to be processed is divided into a time program instruction sub-region corresponding to the upper marker line. The pixel ordinate of the lower end of the time program instruction sub-region is consistent with the pixel ordinate of the end of the upper marker line away from the horizontal time axis, and the two endpoints of the lower end are respectively located at the center of the upper marker line and the two adjacent marker lines. The area below the lower identifier line to the bottom of the timing diagram to be processed is divided into a time program instruction sub-region corresponding to the lower identifier line. The pixel ordinate of the upper end of the time program instruction sub-region is consistent with the pixel ordinate of the end of the lower identifier line away from the horizontal time axis, and the two endpoints of the upper end are respectively located at the center of the lower identifier line and the two adjacent identifier lines. The area between two adjacent marker lines is divided into instruction time interval sub-regions corresponding to the two adjacent marker lines. If the two adjacent marker lines are located on the same side of the horizontal time axis, the four endpoints of the two adjacent marker lines are respectively used as the four endpoints of the instruction time interval sub-region. If the two adjacent marker lines are located on both sides of the horizontal time axis, then the ends of the two adjacent marker lines that are away from the horizontal time axis are taken as the two diagonal endpoints of the instruction time interval sub-region, and the pixel horizontal coordinates of the left and right ends of the time program instruction sub-region are consistent with the pixel horizontal coordinates of the two adjacent marker lines.
5. The method according to claim 1, characterized in that, Before encoding based on the text information within the at least one sub-region and the pixel positions of the at least one sub-region, the method further includes: The text information within each of the at least one sub-region is identified using a text recognition algorithm.
6. A flight timing generation device based on image recognition, characterized in that, The device includes: The acquisition module is used to acquire the original timing diagram and perform grayscale conversion on the original timing diagram to obtain the timing diagram to be processed. The detection module is used to sequentially detect each feature point of the time sequence diagram to be processed, and identify the horizontal time axis and at least one identifier line representing the time program instruction of the time sequence diagram to be processed based on the pixel position and gray value of each feature point, wherein one end of the at least one identifier line is located on the horizontal time axis. The segmentation module is used to divide the time sequence diagram to be processed into at least one sub-region based on the horizontal time axis, the at least one marker line, and the pixel position corresponding to the at least one marker line. The encoding module is used to perform flight timing encoding based on the text information within the at least one sub-region and the pixel positions of the at least one sub-region.
7. The apparatus according to claim 6, characterized in that, The detection module is specifically used for: The image feature point detection algorithm sequentially detects each feature point of the time sequence image to be processed; if a continuous number of feature points with gray values within a preset range are detected, and the pixel ordinates of the number of feature points are the same, then the number of feature points is the horizontal time axis. If a number of consecutive feature points with gray values within a preset range are detected, and the pixel x-coordinates of the number of feature points are the same, then the number of feature points is the marker line.
8. A flight timing generation device based on image recognition, characterized in that, include: A processor and a memory, wherein the memory is used to store a program; The processor is used to execute the program in the memory, causing the computer to perform the image recognition-based flight timing generation method as described in any one of claims 1 to 5.
9. A computer-readable storage medium, characterized in that, It includes computer program instructions that, when run on a computer, cause the computer to perform the image recognition-based flight timing generation method as described in any one of claims 1 to 5.