A computer vision-based intelligent management system for three inspections of a public vehicle

Abstract

The application discloses a kind of based on computer vision's public transport vehicle three inspections intelligent management system, it is related to intelligent traffic and vehicle safety detection technical field, including: according to three inspection phase binding vehicle image, get three inspection phase visual data;Identify vehicle component to be inspected, generate component visual evidence vector;Construct irregular component lattice, get vehicle component visual lattice;Phase mapping update and generate grid point external excitation quantity;Transmit adjacent component state influence, generate vehicle component phase state lattice;Generate three inspection abnormal evolution result;Update phase state lattice, output reinspection feedback update result.The application improves the accuracy of public transport vehicle three inspection abnormal identification, abnormal continuation tracking and reinspection closed-loop verification.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation and vehicle safety inspection technology, and in particular to an intelligent management system for the three-inspection of public transportation vehicles based on computer vision. Background Technology

[0002] In the field of intelligent transportation and vehicle safety inspection technology, the existing three-inspection management of public transportation mainly relies on manual inspection, mini-program reporting, abnormality photo uploading, and stage signature confirmation. It can achieve the retention of inspection records and traceability of responsibilities, but the inspection results mainly depend on the subjective judgment of drivers or inspectors. Image data is usually only saved as an attachment, making it difficult to perform structured identification and continuous analysis of the status of the vehicle's components to be inspected.

[0003] Existing solutions use computer vision to detect targets or identify anomalies in vehicle images to determine conditions such as vehicle body damage, missing parts, and abnormal lighting. However, they typically process a single image or a single part, and the visual recognition results are directly converted into normal or abnormal conclusions. They do not consider pre-departure inspection, in-driving inspection, and post-return inspection as different three-inspection phases in the state update, nor do they convert the adjacent relationships, functional linkage relationships, and the order of the three-inspection relationships between the vehicle's inspected parts into computable coupling edges.

[0004] Existing technologies struggle to establish continuous state evolution relationships among the three inspection phases of the same vehicle's components, making it difficult to effectively identify abnormal enhancement, abnormal weakening, abnormal continuation, and states requiring verification. Furthermore, re-inspection images after repair or abnormal handling are often stored as result evidence, lacking a feedback update mechanism for the phase state grid of vehicle components. This results in insufficient tracking of abnormal evolution, inadequate verification of re-inspection closed loops, and difficulty in collaboratively judging the associated states of components during the three inspection process of public buses.

[0005] Therefore, how to provide a computer vision-based intelligent management system for the three inspections of public transportation vehicles is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a computer vision-based intelligent management system for three-inspection of public transport vehicles. This invention utilizes computer vision recognition and a phase-coupled image lattice model for three-inspection to transform visual evidence of the vehicle's components to be inspected into the grid state of the components and perform cross-phase evolution analysis. It has the advantages of accurate anomaly identification, traceability of the three-inspection process, and reliable closed-loop feedback for re-inspection.

[0007] According to an embodiment of the present invention, a computer vision-based intelligent management system for three inspections of public buses includes the following modules: The phase binding module is used to acquire the three-inspection data of buses, and to perform phase binding on the vehicle image data based on the three-inspection phases in the three-inspection data to obtain the three-inspection phase visual data. The visual evidence module is used to perform computer vision recognition on the three-phase visual data of the inspection, extract the visual evidence corresponding to the vehicle parts to be inspected, and generate the visual evidence vector of the parts. The grid construction module is used to construct irregular component grids according to the association relationship of bus vehicle components. The vehicle component to be inspected is set as component grid points, the association relationship of bus vehicle components is set as coupling edges, and the component visual evidence vector is written into the corresponding component grid points to obtain the vehicle component visual grid. The image excitation module is used to input the visual grid of the vehicle component into the three-detection phase-coupled image grid model, perform phase image update on the component grid points according to the three-detection phase, obtain the phase image state of the component grid points, and generate the external excitation quantity of the grid points according to the visual evidence vector of the component. The coupling propagation module is used to correct the phase image state of the component grid point using the external excitation of the grid point, and to propagate the state influence of adjacent components along the coupling edge to generate the phase state grid of the vehicle component. The evolution analysis module is used to perform cross-phase evolution analysis on the phase state grid of vehicle components, read the grid state of components under different three-inspection phases, extract the abnormal evolution state of components, and generate abnormal evolution results of three-inspection. The re-inspection feedback module is used to collect re-inspection images after anomaly handling, perform computer vision recognition on the re-inspection images, determine the grid points of the components to be updated based on the three-inspection anomaly evolution results, perform feedback update on the phase state grid of the vehicle components, and output the re-inspection feedback update results.

[0008] Optionally, the phase binding module specifically includes: Acquire driving task data, vehicle object data, inspection submission data, and vehicle image data, and integrate the driving task data, vehicle object data, and inspection submission data into three-inspection related information; Extract the three inspection phases corresponding to pre-vehicle inspection, in-vehicle inspection, and post-vehicle inspection from the inspection submission data; Using the three-inspection association information and the three-inspection phase as indexes, vehicle object correspondence and inspection phase correspondence are performed on the vehicle image data to obtain the three-inspection phase visual data.

[0009] Optionally, the visual evidence collection module specifically includes: Read the vehicle image data from the three-inspection phase vision data, locate the vehicle image data according to the vehicle component to be inspected, and obtain the vehicle component to be inspected area. Component identification and image evidence extraction are performed on the area of ​​the vehicle to be inspected to obtain component identification data and area evidence data; Component status extraction and acquisition quality assessment are performed on regional evidence data to obtain status evidence data and acquisition quality data; For the same vehicle component to be inspected, the component identification data, status evidence data, and collection quality data are combined into a component visual evidence vector.

[0010] Optionally, the grid construction module specifically includes: Map the vehicle components to be inspected as component grid points; Based on the adjacent relationships, functional linkage relationships, and three-inspection sequence relationships of the components to be inspected in the bus vehicle structure, generate the bus vehicle component association relationships; Map the relationships between bus vehicle components to the grid points between components, establish coupling edges connecting the grid points, and generate irregular component grids from the grid points and coupling edges; The visual evidence vectors of the components are written into the corresponding component grid points to obtain the visual grid of the vehicle components.

[0011] Optionally, the three-inspection phase-coupled image lattice model has a phase image layer, an evidence excitation layer, a component propagation layer, and a re-inspection feedback layer. The phase image layer performs phase image updates on the component grid points based on the three-inspection phases. The evidence excitation layer generates external excitation quantities acting on the component grid points based on the component visual evidence vectors. The component propagation layer transmits the influence of adjacent component states between component grid points based on the coupling edges. The re-inspection feedback layer performs feedback updates on the vehicle component phase state lattice based on the re-inspection images and the three-inspection anomaly evolution results.

[0012] Optionally, the phase image update of the image excitation module includes the following steps: Based on the three-inspection phase, the phase update path of the component grid is determined, and the state evidence data is extracted from the component visual evidence vector. The state evidence data is then used to determine the current state of the component. The three-phase inspection is performed before vehicle departure, determining the current component status as the inspection start phase image status; The three-phase inspection is performed while the vehicle is in motion. The current component status is compared with the status of the inspection phase image to obtain the vehicle phase image status. The three-phase inspection is performed after the vehicle is stopped. The current component status, the start-up phase image status and the driving phase image status are compared to obtain the stop-up phase image status. Write the phase image state corresponding to the current three detection phases into the component grid points to obtain the component grid point phase image state.

[0013] Optionally, the generation of external excitation quantities at the grid points includes the following steps: The evidence excitation layer obtains state evidence data and acquisition quality data from the component visual evidence vector, and locates the component grid points corresponding to the state evidence data; The state evidence data is matched with the state of the component grid phase image to determine the direction of correction of the state evidence data to the state of the component grid phase image. Based on the collected quality data, determine whether the state evidence data meets the image validity requirements. State evidence data that meets the image validity requirements is retained as valid state evidence data, while state evidence data that does not meet the image validity requirements is excluded from the grid point external excitation quantity generation process. Based on the correction direction corresponding to the valid state evidence data, generate the grid external excitation quantity acting on the component grid.

[0014] Optionally, the coupling propagation module specifically includes: Write the external excitation quantity of the grid point into the corresponding component grid point, and perform excitation correction on the phase image state of the component grid point; Determine the adjacent component grid points connected to the component grid points along the coupling edge, and read the component grid point state after excitation correction from the adjacent component grid points; Based on the association relationship of bus vehicle components corresponding to the coupling edge, the influence of the state of the component grid points after excitation correction of adjacent component grid points is transmitted to obtain the state influence of adjacent components. The component grid points are updated based on the influence of adjacent component states. After the state influence is transferred one by one according to the coupling edge connection relationship, the phase state grid of the vehicle component is obtained.

[0015] Optionally, the evolutionary analysis module specifically includes: Based on the same vehicle component to be inspected, extract the corresponding component grid state from the vehicle component phase state grid for pre-vehicle inspection, in-vehicle inspection, and post-vehicle inspection. Using the grid status of the corresponding components checked before departure as a reference, the grid status of the corresponding components checked during driving is verified to obtain the driving phase evolution result; Using the grid status of the corresponding components during pre-departure and in-driving inspections as reference states, the grid status of the corresponding components during post-departure inspections is verified to obtain the vehicle return phase evolution results. Based on the vehicle phase evolution results and the vehicle termination phase evolution results, enhanced abnormal states, weakened abnormal states, abnormal continuation states, and states to be reviewed are extracted to obtain the abnormal evolution states of the components. Based on the abnormal evolution status of the components to be inspected in the same vehicle, the abnormal evolution results of the three inspections are generated.

[0016] Optionally, the re-inspection feedback module specifically includes: The re-inspection feedback layer determines the grid points of the components to be updated based on the abnormal evolution results of the three inspections; Computer vision recognition is performed on the re-inspection images after anomaly handling to extract the re-inspection visual evidence corresponding to the grid points of the component to be updated. The visual evidence of re-inspection is matched with the state of the component grid point in the phase state grid of the vehicle component to be updated, and the re-inspection state matching result is obtained, and the external excitation quantity of re-inspection is generated. The external excitation quantity for re-inspection is written into the grid point of the component to be updated, the phase state grid of the vehicle component is updated, and the re-inspection feedback update result is obtained.

[0017] The beneficial effects of this invention are: (1) This invention binds vehicle image data through three-inspection phases and extracts the component visual evidence vector corresponding to the vehicle component to be inspected. The images of the pre-departure inspection, in-driving inspection and post-return inspection are no longer at the level of manual traces, but are transformed into identifiable, callable and traceable structured three-inspection data.

[0018] (2) This invention transforms the vehicle components to be inspected into component grid points by using irregular component grids, transforms the association relationship of bus components into coupling edges, and performs state updates by combining the external excitation quantity of grid points and the state influence of adjacent components, thereby improving the accuracy of identifying abnormalities, enhancements and continuations of associated components.

[0019] (3) This invention generates the three-inspection anomaly evolution results through cross-phase evolution analysis. After anomaly handling, the phase state grid of vehicle components is updated by using the re-inspection image. Anomaly detection, state evolution, and re-inspection confirmation form a closed loop, reducing the problems of manual missed detection, wrong detection, and insufficient re-inspection basis. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of a computer vision-based intelligent management system for three inspections of public buses proposed in this invention. Figure 2 This is a schematic diagram of the structure of the three-detector phase-coupled image lattice model in this invention; Figure 3 This is a schematic diagram illustrating the construction of irregular component grids and vehicle component visual grids in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] refer to Figures 1-3 A computer vision-based intelligent management system for the three-inspection of public transport vehicles includes the following modules: The phase binding module is used to acquire the three-inspection data of buses, and to perform phase binding on the vehicle image data based on the three-inspection phases in the three-inspection data to obtain the three-inspection phase visual data. The visual evidence module is used to perform computer vision recognition on the three-phase visual data of the inspection, extract the visual evidence corresponding to the vehicle parts to be inspected, and generate the visual evidence vector of the parts. The grid construction module is used to construct irregular component grids according to the association relationship of bus vehicle components. The vehicle component to be inspected is set as component grid points, the association relationship of bus vehicle components is set as coupling edges, and the component visual evidence vector is written into the corresponding component grid points to obtain the vehicle component visual grid. The image excitation module is used to input the visual grid of the vehicle component into the three-detection phase-coupled image grid model, perform phase image update on the component grid points according to the three-detection phase, obtain the phase image state of the component grid points, and generate the external excitation quantity of the grid points according to the visual evidence vector of the component. The coupling propagation module is used to correct the phase image state of the component grid point using the external excitation of the grid point, and to propagate the state influence of adjacent components along the coupling edge to generate the phase state grid of the vehicle component. The evolution analysis module is used to perform cross-phase evolution analysis on the phase state grid of vehicle components, read the grid state of components under different three-inspection phases, extract the abnormal evolution state of components, and generate abnormal evolution results of three-inspection. The re-inspection feedback module is used to collect re-inspection images after anomaly handling, perform computer vision recognition on the re-inspection images, determine the grid points of the components to be updated based on the three-inspection anomaly evolution results, perform feedback update on the phase state grid of the vehicle components, and output the re-inspection feedback update results.

[0023] In this embodiment, the phase binding module specifically includes: Acquire driving task data, vehicle object data, inspection submission data, and vehicle image data, and integrate the driving task data, vehicle object data, and inspection submission data into three-inspection related information; Driving task data, vehicle object data, and inspection submission data are aggregated into the same three-inspection batch. During aggregation, the logged-in driver, current driving task, current vehicle, and submission action are read. First, the attribution relationship between the logged-in driver and the current driving task is verified, then the attribution relationship between the current vehicle and the current driving task is verified. The submission action is then assigned to the corresponding three-inspection batch. The verified data is used to synthesize the three-inspection association information, and the vehicle image data is retained as a phase-binding object.

[0024] Extract the three inspection phases corresponding to pre-vehicle inspection, in-vehicle inspection, and post-vehicle inspection from the inspection submission data; The inspection process reads and checks the data entry and submission nodes. The inspection entry is categorized into three types: before the vehicle, during driving, and after vehicle retrieval. The submission node records the submission action at the current entry. When a submission node passes, a valid phase marker is generated; when a submission node fails, the vehicle image data enters a temporary storage set and is not included in the three-inspection phase visual data.

[0025] Using the three-inspection association information and the three-inspection phase as indexes, vehicle object correspondence and inspection phase correspondence are performed on the vehicle image data to obtain the three-inspection phase visual data.

[0026] The three-inspection association information is used as the basis for verifying the attribution of vehicle image data, determining the driving task, vehicle object, and submission batch corresponding to the vehicle image data; the three-inspection phases are used as the basis for classifying the inspection stages, writing the vehicle image data into the matching phase sets in the pre-departure inspection, in-driving inspection, and post-return inspection. Vehicle image data, three-inspection association information, and three-inspection phases together constitute the three-inspection phase visual data.

[0027] In this embodiment, the visual evidence collection module specifically includes: Read the vehicle image data from the three-inspection phase vision data, locate the vehicle image data according to the vehicle component to be inspected, and obtain the vehicle component to be inspected area. Read the three-phase inspection data and determine the vehicle component to be inspected according to the three-phase inspection data; retrieve the image region corresponding to the vehicle component to be inspected from the vehicle image data; perform component name matching and image position verification on the retrieved image region, and retain the image region that passes the matching and verification as the vehicle component to be inspected region.

[0028] Component identification and image evidence extraction are performed on the area of ​​the vehicle to be inspected to obtain component identification data and area evidence data; The outline and installation location features of the vehicle component to be inspected are extracted to obtain component identification data; image segments corresponding to the vehicle component to be inspected are extracted and the regional position of the image segments in the vehicle image data is recorded to obtain regional evidence data.

[0029] Component status extraction and acquisition quality assessment are performed on regional evidence data to obtain status evidence data and acquisition quality data; Image fragments are read from the regional evidence data, and the status items of the vehicle components to be inspected are extracted to obtain status evidence data. The occlusion status, imaging clarity status, and shooting angle offset status in the image fragments are read to obtain acquisition quality data. The acquisition quality data is used to correct the conversion process of status evidence data to external excitation quantities of grid points.

[0030] For the same vehicle component to be inspected, the component identification data, status evidence data, and collection quality data are combined into a component visual evidence vector.

[0031] The component identification data, status evidence data, and acquisition quality data are merged according to the three-inspection association information, the three-inspection phase, and the vehicle component to be inspected. Data under the same vehicle object, the same three-inspection phase, and the same vehicle component to be inspected are written into the same component visual evidence vector. The component identification data is used to match the component grid points, and the status evidence data and acquisition quality data are used to generate external excitation quantities for grid points in the future.

[0032] In this embodiment, the grid construction module specifically includes: Map the vehicle components to be inspected as component grid points; Based on the previously determined vehicle components to be inspected, component grid points are established one by one, and the vehicle components to be inspected are bound to the component grid points. The component grid points serve as the state-carrying positions corresponding to the vehicle components to be inspected, are used to receive the component visual evidence vectors, and serve as the connection endpoints of coupling edges.

[0033] Based on the adjacent relationships, functional linkage relationships, and three-inspection sequence relationships of the components to be inspected in the bus vehicle structure, generate the bus vehicle component association relationships; The installation proximity relationships between the components to be inspected are extracted from the bus structure records, and the functional influence relationships and verification sequence relationships between the components to be inspected are extracted from the three-inspection item rules. The installation proximity relationships, functional influence relationships, and verification sequence relationships are combined into the bus component association relationships.

[0034] Map the relationships between bus vehicle components to the grid points between components, establish coupling edges connecting the grid points, and generate irregular component grids from the grid points and coupling edges; The vehicle components to be inspected in the bus component association relationship are replaced with the corresponding component grid points, generating grid point connection relationships. Coupled edges are established based on these grid point connection relationships, retaining their corresponding association types for subsequent use when propagating the state influence of adjacent components along the coupled edges. Component grid points and coupled edges form irregular component grids, which are connected according to the bus component association relationships.

[0035] The visual evidence vectors of the components are written into the corresponding component grid points to obtain the visual grid of the vehicle components.

[0036] According to the vehicle component to be inspected corresponding to the component's visual evidence vector, the component's visual evidence vector is written into the corresponding component grid point. After writing, the component grid point carries the component's visual evidence vector, and the coupling edge carries the association type, resulting in the vehicle component visual grid. The vehicle component visual grid serves as the input for the subsequent three-inspection phase-coupled mapping grid model.

[0037] In this embodiment, the three-inspection phase-coupled image lattice model has a phase image layer, an evidence excitation layer, a component propagation layer, and a re-inspection feedback layer. The phase image layer performs phase image updates on the component grid points based on the three-inspection phases. The evidence excitation layer generates external excitation quantities for the grid points acting on the component grid points based on the component visual evidence vectors. The component propagation layer transmits the influence of adjacent component states between component grid points based on the coupling edges. The re-inspection feedback layer performs feedback updates on the vehicle component phase state lattice based on the re-inspection images and the three-inspection anomaly evolution results.

[0038] In this embodiment, the phase image update of the image excitation module includes the following steps: Based on the three-inspection phase, the phase update path of the component grid is determined, and the state evidence data is extracted from the component visual evidence vector. The state evidence data is then used to determine the current state of the component. After receiving the visual evidence vector of a component, the phase image layer locates the component grid points according to the component identification data, obtains state evidence data from the same component's visual evidence vector, and uses the state evidence data as the current component state. The three-inspection phase control update path is as follows: pre-departure inspection enters the start-up update path, in-vehicle inspection enters the change comparison path, and post-retirement inspection enters the continuation comparison path. Acquired quality data does not participate in the phase image update; it is left to the evidence excitation layer to generate external excitation quantities for the grid points.

[0039] The three-phase inspection is performed before vehicle departure, determining the current component status as the inspection start phase image status; In the inspection update path, the current component status serves as the phase reference for the component to be inspected in the same vehicle and is saved as the inspection phase image status. The inspection phase image status retains the correspondence between the component to be inspected in the vehicle and the pre-departure inspection, which can be called upon for in-vehicle inspection and post-return inspection.

[0040] The three-phase inspection is performed while the vehicle is in motion. The current component status is compared with the status of the inspection phase image to obtain the vehicle phase image status. In the change comparison path, the current component status is compared with the start-up phase image status of the component to be inspected on the same vehicle to identify the status changes from pre-vehicle inspection to in-vehicle inspection. The status change results are saved as driving phase image status for continued use after vehicle return inspection.

[0041] The three-phase inspection is performed after the vehicle is stopped. The current component status, the start-up phase image status and the driving phase image status are compared to obtain the stop-up phase image status. In the continuous comparison path, the current component status is compared with the inspection start phase image status and the driving phase image status to identify the changes in status from pre-departure inspection to driving inspection, and from driving inspection to post-retirement inspection. The changes are saved as the retirement phase image status for cross-phase evolution analysis.

[0042] Write the phase image state corresponding to the current three detection phases into the component grid points to obtain the component grid point phase image state.

[0043] Select the corresponding phase image state according to the current three-inspection phase. The current three-inspection phase is the phase image state written during the pre-departure inspection, the phase image state written during the in-vehicle inspection, and the phase image state written during the post-retirement inspection. After writing, the component grid phase image state is obtained, which is then entered into the evidence excitation layer and the component propagation layer.

[0044] In this embodiment, the generation of external excitation quantities for grid points includes the following steps: The evidence excitation layer obtains state evidence data and acquisition quality data from the component visual evidence vector, and locates the component grid points corresponding to the state evidence data; After receiving the visual evidence vectors from the components, the evidence excitation layer determines the corresponding grid points for each component to be inspected on the vehicle. Component identification data is used to complete grid point localization, state evidence data is used for state consistency matching, and acquisition quality data is used for image validity requirement determination. The three types of data are then processed at their respective processing locations.

[0045] The state evidence data is matched with the state of the component grid phase image to determine the direction of correction of the state evidence data to the state of the component grid phase image. State consistency matching first retrieves the phase image state of the component grid points, and then retrieves the state evidence data of the component under inspection from the same vehicle. When the two states are consistent, the correction direction maintains the phase image state of the component grid points; when the state evidence data shows an enhanced anomalous state, the correction direction points to the enhanced anomalous state; when the state evidence data shows a weakened anomalous state, the correction direction points to the weakened anomalous state; when the state evidence data cannot support state consistency matching, the correction direction points to the state to be reviewed.

[0046] Based on the collected quality data, determine whether the state evidence data meets the image validity requirements. State evidence data that meets the image validity requirements is retained as valid state evidence data, while state evidence data that does not meet the image validity requirements is excluded from the grid point external excitation quantity generation process. The validity requirements for an image are determined by the quality of the acquired data. If the acquired data indicates that the inspected component of the vehicle is obstructed, blurred, or has an angular offset, affecting state consistency matching, the corresponding state evidence data will not be included in the external excitation generation process for the current grid point. If the acquired quality data can support state consistency matching, the corresponding state evidence data will be retained as valid state evidence data.

[0047] Based on the correction direction corresponding to the valid state evidence data, generate the grid external excitation quantity acting on the component grid.

[0048] The evidence excitation layer binds valid state evidence data with the corresponding correction direction, generating grid external excitation quantities that act on the corresponding component grid points. These grid external excitation quantities are written to the component grid points mapped from the vehicle component under inspection to which the valid state evidence data belongs, and are used for subsequent correction of the component grid point phase image state.

[0049] In this embodiment, the coupling propagation module specifically includes: Write the external excitation quantity of the grid point into the corresponding component grid point, and perform excitation correction on the phase image state of the component grid point; After the external excitation quantity enters the component grid, it first performs state consistency matching with the phase mapping state of the component grid. When the external excitation quantity points to the hold state, the component grid phase mapping state is retained; when it points to the enhanced abnormal state, the component grid state is adjusted to the enhanced abnormal state; when it points to the weakened abnormal state, the component grid state is adjusted to the weakened abnormal state; when it points to the pending review state, the component grid state enters the pending review state.

[0050] Determine the adjacent component grid points connected to the component grid points along the coupling edge, and read the component grid point state after excitation correction from the adjacent component grid points; Starting with the component grid point that has completed excitation correction, the component grid point connected to the other end is read along the coupling edge. The component grid point connected to the other end is regarded as the adjacent component grid point. The component propagation layer reads the state of the component grid point after the excitation correction of the adjacent component grid point as the input for subsequent influence propagation.

[0051] Based on the association relationship of bus vehicle components corresponding to the coupling edge, the influence of the state of the component grid points after excitation correction of adjacent component grid points is transmitted to obtain the state influence of adjacent components. The component propagation layer reads the association type carried by the coupling edge. When the association type is an adjacency relationship, it transmits changes in appearance status; when the association type is a functional linkage relationship, it transmits changes in safety function status; when the association type is a three-inspection sequence relationship, it transmits the status succession relationship corresponding to the three-inspection sequence relationship. The transmission result is recorded as the state influence of adjacent components.

[0052] The component grid points are updated based on the influence of adjacent component states. After the state influence is transferred one by one according to the coupling edge connection relationship, the phase state grid of the vehicle component is obtained.

[0053] After receiving the state influence of adjacent components, the component grid point first checks the direction of the adjacent component state influence with the component grid point state. If the directions are consistent, the corresponding state is strengthened; if the directions conflict, it enters a state awaiting verification; if there is no effective state influence from adjacent components, the component grid point state after excitation correction is retained. After the state influence is transmitted one by one according to the coupling edge connection relationship, the updated component grid point and coupling edge form the vehicle component phase state grid.

[0054] In this embodiment, the evolution analysis module specifically includes: Based on the same vehicle component to be inspected, extract the corresponding component grid state from the vehicle component phase state grid for pre-vehicle inspection, in-vehicle inspection, and post-vehicle inspection. Cross-phase evolution analysis identifies the vehicle object from the phase state grid of vehicle components based on the three-inspection association information, and then aggregates the corresponding component grid states for pre-departure inspection, in-vehicle inspection, and post-inspection according to the vehicle components to be inspected. When the three-inspection phase lacks component grid states, the corresponding vehicle component to be inspected is transferred to the pending review state and does not participate in the extraction of enhanced or weakened abnormal states.

[0055] Using the grid status of the corresponding components checked before departure as a reference, the grid status of the corresponding components checked during driving is verified to obtain the driving phase evolution result; Using the pre-departure check of the corresponding component grid status as a reference status, the in-vehicle check of the corresponding component grid status is verified. If the in-vehicle check of the corresponding component grid status indicates an increased aberration relative to the reference status, an enhanced aberration status is obtained; if the aberration is reduced, a weakened aberration status is obtained; if the phase status cannot be matched, a status pending verification is obtained and written into the vehicle phase evolution result.

[0056] Using the grid status of the corresponding components during pre-departure and in-driving inspections as reference states, the grid status of the corresponding components during post-departure inspections is verified to obtain the vehicle return phase evolution results. Using the grid status of the components corresponding to the pre-departure and in-vehicle inspections as reference states, the grid status of the components corresponding to the post-receipt inspection is verified. When the grid status of the components corresponding to the post-receipt inspection is consistent with the previous abnormal state, an abnormality continuation state is obtained; when the abnormality is reduced relative to the previous abnormal state, a weakened abnormal state is obtained; when it conflicts with the previous phase state, a state pending verification is obtained and written into the post-receipt phase evolution result.

[0057] Based on the vehicle phase evolution results and the vehicle termination phase evolution results, enhanced abnormal states, weakened abnormal states, abnormal continuation states, and states to be reviewed are extracted to obtain the abnormal evolution states of the components. Enhanced abnormal states, weakened abnormal states, and states awaiting review are extracted from the driving phase evolution results. Abnormal continuation states, weakened abnormal states, and states awaiting review are extracted from the vehicle termination phase evolution results. When multiple states exist for the same vehicle's inspected component, the abnormal change process is retained according to the order of the three inspection phases to generate the component's abnormal evolution state.

[0058] Based on the abnormal evolution status of the components to be inspected in the same vehicle, the abnormal evolution results of the three inspections are generated.

[0059] The driving phase evolution results, vehicle termination phase evolution results, and component abnormal evolution states corresponding to the same vehicle component under inspection are grouped into the same three-inspection abnormal evolution result. The three-inspection abnormal evolution result retains the vehicle component under inspection and the source of the three-inspection phases, which is used by the re-inspection feedback layer to determine the grid points of the component to be updated.

[0060] In this embodiment, the re-inspection feedback module specifically includes: The re-inspection feedback layer determines the grid points of the components to be updated based on the abnormal evolution results of the three inspections; The re-inspection feedback layer reads the component abnormality evolution status from the three-inspection abnormality evolution results, filters out the vehicle components to be inspected corresponding to enhanced abnormality status, abnormality continuation status, and pending review status; then it locates the component grid points according to the vehicle components to be inspected, as the component grid points to be updated in the feedback. Weakening abnormality statuses only retain their evolution records and are not included in this feedback update.

[0061] Computer vision recognition is performed on the re-inspection images after anomaly handling to extract the re-inspection visual evidence corresponding to the grid points of the component to be updated. Before the re-inspection image enters the visual recognition, the recognition area of ​​the vehicle to be inspected component is limited according to the grid point of the component to be updated. The visual recognition only extracts the re-inspection visual evidence of the same vehicle to be inspected component within the limited area, and the re-inspection visual evidence enters the state consistency matching corresponding to the grid point of the component to be updated.

[0062] The visual evidence of re-inspection is matched with the state of the component grid point in the phase state grid of the vehicle component to be updated, and the re-inspection state matching result is obtained, and the external excitation quantity of re-inspection is generated. The re-inspection feedback layer performs state consistency matching between the re-inspection visual evidence and the state of the component grid points to be updated in the vehicle component phase state grid. If the re-inspection visual evidence still points to an anomaly, an anomaly preservation result is obtained; if it points to an anomaly mitigation result, an anomaly mitigation result is obtained; if the re-inspection visual evidence cannot achieve state consistency matching, a result awaiting review is obtained. Based on the anomaly preservation result, the anomaly mitigation result, and the result awaiting review, an external excitation quantity for re-inspection is generated.

[0063] The external excitation quantity for re-inspection is written into the grid point of the component to be updated, the phase state grid of the vehicle component is updated, and the re-inspection feedback update result is obtained.

[0064] After the external excitation quantity for re-inspection is written into the grid point of the component to be updated, the re-inspection feedback layer updates the state of the corresponding component grid point in the phase state grid of the vehicle component. After the update is completed, the grid point of the component to be updated, the re-inspection state matching result, and the updated component grid point state are recorded to obtain the re-inspection feedback update result.

[0065] Example 1: To verify the feasibility of this invention in practice, it was applied to a daily three-inspection management scenario for a public bus. In this scenario, drivers need to complete vehicle safety checks in three stages: before departure, during operation, and after return. The checks cover all components of the vehicle to be inspected. Conventional single-frame visual recognition three-inspection methods can identify components and determine anomalies in uploaded images, but they typically only make a single recognition conclusion for the current image. There is a lack of continuous state transition between the image recognition results and the three inspection stages, and there is also a lack of correlation analysis between the components of the vehicle to be inspected. This makes it difficult to reliably track the changes of the same anomaly in the three stages: before departure, during operation, and after return.

[0066] In this scenario, the driver still follows their original operating habits to access the three-inspection interface, select the driving task and vehicle object, and then upload the vehicle image. After acquiring the bus's three-inspection data, phase binding is performed on the vehicle image data according to the three-inspection phases, converting the vehicle images corresponding to the pre-departure inspection, in-driving inspection, and post-arrival inspection into three-inspection phase visual data. The visual evidence module performs computer vision recognition on the three-inspection phase visual data, extracts the visual evidence corresponding to the vehicle's inspected components, and generates component visual evidence vectors. The vehicle image is no longer just an uploaded certificate, but is transformed into structured visual data that can be used for model calculation.

[0067] The system maps vehicle components to be inspected as component grid points. Based on the adjacency, functional linkage, and inspection sequence of these components within the bus structure, it generates component relationships. These relationships are then mapped to the grid points, establishing coupling edges. Irregular component grids are generated from these grid points and coupling edges. After the visual evidence vectors of the components are written to the corresponding grid points, the visual grids of the vehicle components are obtained. Vehicle components to be inspected, such as doors, sides, lights, tires, and safety features, are no longer identified in isolation but are organized into state grids with interconnected relationships.

[0068] In the three-detection phase-coupled mapping lattice model, the mapping excitation module inputs the vehicle component visual lattice into the model, performs phase mapping updates on the component lattice points based on the three-detection phases, obtains the component lattice point phase mapping state, and generates external excitation quantities for the lattice points based on the component visual evidence vectors. The coupling propagation module uses the external excitation quantities to correct the component lattice point phase mapping state, propagates the influence of adjacent component states along the coupling edges, and generates the vehicle component phase state lattice.

[0069] During cross-phase evolution analysis, the system reads the component grid state under different three-inspection phases from the vehicle component phase state grid, extracts the component's abnormal evolution state, and generates the three-inspection abnormal evolution result. If a component under inspection is normal before departure, an abnormality is found during driving, and the abnormality remains after the vehicle is returned to service, the system will identify it as a continued abnormality state; if the abnormality state lessens after the vehicle is returned to service, it will be identified as a weakened abnormality state; if the states under different phases cannot be stably correlated, it will enter a state awaiting review. This solves the problem that conventional single-frame recognition can only judge the current image and is difficult to track abnormal changes.

[0070] After anomaly handling, the system acquires re-inspection images and performs computer vision recognition on them. The re-inspection feedback module determines the component grid points to be updated based on the anomaly evolution results from the three inspections. It then performs state consistency matching between the re-inspection image recognition results and the component grid point states in the vehicle component phase state grid, generating external excitation quantities for re-inspection and performing feedback updates on the vehicle component phase state grid. Thus, the re-inspection images are no longer merely evidence of anomaly handling, but can be reapplied to the vehicle component state grid to verify whether the anomaly still exists, whether it has been mitigated, and whether further verification is needed.

[0071] The conventional single-frame visual recognition three-inspection method was used as a control, and the system of this invention was used as the implementation method. Both methods used the same vehicle image source and the same manual review conclusion as the statistical basis, comparing image matching accuracy, anomaly recognition accuracy, anomaly continuation recognition rate, re-inspection confirmation accuracy, and single-vehicle three-inspection time. Specific results are shown in Table 1.

[0072] Table 1. Comparison of the effects of intelligent management of the three inspections of public buses

[0073] As shown in Table 1, compared with the conventional single-frame visual recognition three-inspection method, the system of the present invention improves in five aspects: image matching accuracy, anomaly recognition accuracy, anomaly continuation recognition rate, re-inspection confirmation accuracy, and single-vehicle three-inspection time. Among them, the image matching accuracy increased from 88.9% to 96.8%, indicating that the three-inspection phase binding and vehicle inspection component positioning can reduce the mismatch between images and inspection objects. Vehicle images in the three stages of pre-departure, during-driving, and post-return can be more stably mapped to specific vehicle inspection components, providing a more reliable data foundation for subsequent component visual evidence vector generation and grid state update.

[0074] The anomaly identification accuracy improved from 85.6% to 94.6%, mainly due to the fact that this invention does not directly use the identification result of a single frame image as the final anomaly conclusion. Instead, it writes the component visual evidence vector into the component grid and corrects the phase image state of the component grid by combining the external excitation quantity of the grid. This reduces the risk of misjudgment caused by occlusion, shooting angle deviation, and insufficient imaging quality of a single image, making the judgment of the status of the vehicle's inspected components more stable. The anomaly continuation identification rate improved from 72.4% to 91.3%, showing the largest increase. This indicates that irregular component grids and cross-phase evolution analysis have a significant effect on anomaly tracking in the continuous three-stage inspection process, and can identify the changes in anomalies from inspection during driving to inspection after vehicle return, and from inspection before departure to subsequent stages.

[0075] The accuracy rate of re-inspection confirmation increased from 81.5% to 93.7%, indicating that the re-inspection feedback update mechanism of this invention can overcome the shortcomings of conventional methods where re-inspection images are only stored as evidence. After the re-inspection images are recognized by computer vision, a re-inspection conclusion is not formed independently. Instead, the grid points of the components to be updated are determined based on the evolution results of the three-inspection anomalies, and feedback updates are performed on the phase state grids of the vehicle components. Therefore, it is possible to more accurately determine whether the status has improved after anomaly handling, whether it still exists, and whether further review is needed. The time for single-vehicle three-inspection is reduced from 10.9 minutes to 7.8 minutes, indicating that while improving recognition accuracy, this invention also reduces the time spent on manual image verification, finding the source of anomalies, and confirming re-inspection results, thereby improving the efficiency of three-inspection management for public transportation vehicles.

[0076] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A computer vision-based intelligent management system for the three-inspection process of public buses, characterized in that, include: The phase binding module is used to acquire the three-inspection data of buses, and to perform phase binding on the vehicle image data based on the three-inspection phases in the three-inspection data to obtain the three-inspection phase visual data. The visual evidence module is used to perform computer vision recognition on the three-phase visual data of the inspection, extract the visual evidence corresponding to the vehicle parts to be inspected, and generate the visual evidence vector of the parts. The grid construction module is used to construct irregular component grids according to the association relationship of bus vehicle components. The vehicle component to be inspected is set as component grid points, the association relationship of bus vehicle components is set as coupling edge, and the component visual evidence vector is written into the corresponding component grid point to obtain the vehicle component visual grid. The image excitation module is used to input the visual grid of the vehicle component into the three-detection phase-coupled image grid model, perform phase image update on the component grid points according to the three-detection phase, obtain the phase image state of the component grid points, and generate the external excitation quantity of the grid points according to the visual evidence vector of the component. The coupling propagation module is used to correct the phase image state of the component grid point using the external excitation of the grid point, and to propagate the state influence of adjacent components along the coupling edge to generate the phase state grid of the vehicle component. The evolution analysis module is used to perform cross-phase evolution analysis on the phase state grid of vehicle components, read the grid state of components under different three-inspection phases, extract the abnormal evolution state of components, and generate abnormal evolution results of three-inspection. The re-inspection feedback module is used to collect re-inspection images after anomaly handling, perform computer vision recognition on the re-inspection images, determine the grid points of the components to be updated based on the three-inspection anomaly evolution results, perform feedback update on the phase state grid of the vehicle components, and output the re-inspection feedback update results.

2. The intelligent management system for three inspections of public buses based on computer vision according to claim 1, characterized in that, The phase binding module specifically includes: Acquire driving task data, vehicle object data, inspection submission data, and vehicle image data, and integrate the driving task data, vehicle object data, and inspection submission data into three-inspection related information; Extract the three inspection phases corresponding to pre-vehicle inspection, in-vehicle inspection, and post-vehicle inspection from the inspection submission data; Using the three-inspection association information and the three-inspection phase as indexes, vehicle object correspondence and inspection phase correspondence are performed on the vehicle image data to obtain the three-inspection phase visual data.

3. The intelligent management system for three inspections of public buses based on computer vision according to claim 1, characterized in that, The visual evidence collection module specifically includes: Read the vehicle image data from the three-inspection phase vision data, locate the vehicle image data according to the vehicle component to be inspected, and obtain the vehicle component to be inspected area. Component identification and image evidence extraction are performed on the area of ​​the vehicle to be inspected to obtain component identification data and area evidence data; Component status extraction and acquisition quality assessment are performed on regional evidence data to obtain status evidence data and acquisition quality data; For the same vehicle component to be inspected, the component identification data, status evidence data, and collection quality data are combined into a component visual evidence vector.

4. The intelligent management system for three inspections of public buses based on computer vision according to claim 1, characterized in that, The grid construction module specifically includes: Map the vehicle components to be inspected as component grid points; Based on the adjacent relationships, functional linkage relationships, and three-inspection sequence relationships of the components to be inspected in the bus vehicle structure, generate the bus vehicle component association relationships; Map the relationships between bus vehicle components to the grid points between components, establish coupling edges connecting the grid points, and generate irregular component grids from the grid points and coupling edges; The visual evidence vectors of the components are written into the corresponding component grid points to obtain the visual grid of the vehicle components.

5. The intelligent management system for three inspections of public buses based on computer vision according to claim 1, characterized in that, The three-inspection phase-coupled image lattice model has a phase image layer, an evidence excitation layer, a component propagation layer, and a re-inspection feedback layer. The phase image layer updates the phase image of the component grid points based on the three-inspection phase. The evidence excitation layer generates external excitation quantities for the grid points acting on the component grid points based on the component visual evidence vector. The component propagation layer transmits the influence of adjacent component states between component grid points based on the coupling edges. The re-inspection feedback layer updates the phase state grid of the vehicle components based on the re-inspection image and the three-inspection anomaly evolution results.

6. The intelligent management system for three inspections of public buses based on computer vision according to claim 1, characterized in that, The phase image update of the image excitation module includes the following steps: Based on the three-inspection phase, the phase update path of the component grid is determined, and the state evidence data is extracted from the component visual evidence vector. The state evidence data is then used to determine the current state of the component. The three-phase inspection is performed before vehicle departure, determining the current component status as the inspection start phase image status; The three-phase inspection is performed while the vehicle is in motion. The current component status is compared with the status of the inspection phase image to obtain the vehicle phase image status. The three-phase inspection is performed after the vehicle is stopped. The current component status, the start-up phase image status and the driving phase image status are compared to obtain the stop-up phase image status. Write the phase image state corresponding to the current three detection phases into the component grid points to obtain the component grid point phase image state.

7. The intelligent management system for three inspections of public buses based on computer vision according to claim 6, characterized in that, The generation of the external excitation quantity of the grid point includes the following steps: The evidence excitation layer obtains state evidence data and acquisition quality data from the component visual evidence vector, and locates the component grid points corresponding to the state evidence data; The state evidence data is matched with the state of the component grid phase image to determine the direction of correction of the state evidence data to the state of the component grid phase image. Based on the collected quality data, determine whether the state evidence data meets the image validity requirements. State evidence data that meets the image validity requirements is retained as valid state evidence data, while state evidence data that does not meet the image validity requirements is excluded from the grid point external excitation quantity generation process. Based on the correction direction corresponding to the valid state evidence data, generate the grid external excitation quantity acting on the component grid.

8. The intelligent management system for three inspections of public buses based on computer vision according to claim 1, characterized in that, The coupling propagation module specifically includes: Write the external excitation quantity of the grid point into the corresponding component grid point, and perform excitation correction on the phase image state of the component grid point; Determine the adjacent component grid points connected to the component grid points along the coupling edge, and read the component grid point state after excitation correction from the adjacent component grid points; Based on the association relationship of bus vehicle components corresponding to the coupling edge, the influence of the state of the component grid points after excitation correction of adjacent component grid points is transmitted to obtain the state influence of adjacent components. The component grid points are updated based on the influence of adjacent component states. After the state influence is transferred one by one according to the coupling edge connection relationship, the phase state grid of the vehicle component is obtained.

9. The intelligent management system for three inspections of public buses based on computer vision according to claim 1, characterized in that, The evolutionary analysis module specifically includes: Based on the same vehicle component to be inspected, extract the corresponding component grid state from the vehicle component phase state grid for pre-vehicle inspection, in-vehicle inspection, and post-vehicle inspection. Using the grid status of the corresponding components checked before departure as a reference, the grid status of the corresponding components checked during driving is verified to obtain the driving phase evolution result; Using the grid status of the corresponding components during pre-departure and in-driving inspections as reference states, the grid status of the corresponding components during post-departure inspections is verified to obtain the vehicle return phase evolution results. Based on the vehicle phase evolution results and the vehicle termination phase evolution results, enhanced abnormal states, weakened abnormal states, abnormal continuation states, and states to be reviewed are extracted to obtain the abnormal evolution states of the components. Based on the abnormal evolution status of the components to be inspected in the same vehicle, the abnormal evolution results of the three inspections are generated.

10. The intelligent management system for three inspections of public buses based on computer vision according to claim 1, characterized in that, The re-inspection feedback module specifically includes: The re-inspection feedback layer determines the grid points of the components to be updated based on the abnormal evolution results of the three inspections; Computer vision recognition is performed on the re-inspection images after anomaly handling to extract the re-inspection visual evidence corresponding to the grid points of the component to be updated. The re-inspection visual evidence is matched with the state of the component grid point in the vehicle component phase state grid to obtain the re-inspection state matching result and generate the re-inspection external stimulus quantity. The external excitation quantity for re-inspection is written into the grid point of the component to be updated, the phase state grid of the vehicle component is updated, and the re-inspection feedback update result is obtained.

Images